July 2011 - teiid-commits - Jboss List Archives

teiid SVN: r3326 - in branches/7.4.x: client/src/main/java/org/teiid/jdbc and 4 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-22 11:32:56 -0400 (Fri, 22 Jul 2011) New Revision: 3326 Added: branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testColumnMetadataWithAlias.expected Modified: branches/7.4.x/build/kits/jboss-container/teiid-releasenotes.html branches/7.4.x/client/src/main/java/org/teiid/jdbc/ResultSetMetaDataImpl.java branches/7.4.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml branches/7.4.… [View More]x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-extensions.xml branches/7.4.x/runtime/src/main/java/org/teiid/odbc/ODBCServerRemoteImpl.java branches/7.4.x/test-integration/common/src/test/java/org/teiid/transport/TestODBCSocketTransport.java branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPk.expected branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPkPrepared.expected Log: TEIID-1651 hopefully the final touches to jdbc/odbc metadata related to column name/label Modified: branches/7.4.x/build/kits/jboss-container/teiid-releasenotes.html =================================================================== --- branches/7.4.x/build/kits/jboss-container/teiid-releasenotes.html 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/build/kits/jboss-container/teiid-releasenotes.html 2011-07-22 15:32:56 UTC (rev 3326) @@ -75,6 +75,7 @@ <h4>from 7.4</h4> <ul> <li>ColumnReference.getName will always return just the element name. Previously it inconsistently returned the qualified and unqualified form depending upon where the ColumnReference appeared. + <li>As per JDBC4, ResultSetMetadata.getColumnName will return the unaliased column name if available rather than return the alias. Set useJDBC4ColumnNameAndLabelSemantics to false to use the alias name as the column name. </ul> <h4>from 7.3</h4> Modified: branches/7.4.x/client/src/main/java/org/teiid/jdbc/ResultSetMetaDataImpl.java =================================================================== --- branches/7.4.x/client/src/main/java/org/teiid/jdbc/ResultSetMetaDataImpl.java 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/client/src/main/java/org/teiid/jdbc/ResultSetMetaDataImpl.java 2011-07-22 15:32:56 UTC (rev 3326) @@ -35,11 +35,13 @@ private MetadataProvider provider; - private boolean useJDBC4ColumnNameAndLabelSemantics = false; + private boolean useJDBC4ColumnNameAndLabelSemantics = true; public ResultSetMetaDataImpl(MetadataProvider provider, String supportBackwardsCompatibility) { this.provider = provider; - this.useJDBC4ColumnNameAndLabelSemantics = (supportBackwardsCompatibility != null && supportBackwardsCompatibility.equalsIgnoreCase("false") ? false : true); + if (supportBackwardsCompatibility != null) { + this.useJDBC4ColumnNameAndLabelSemantics = Boolean.parseBoolean(supportBackwardsCompatibility); + } } /** Modified: branches/7.4.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml =================================================================== --- branches/7.4.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml 2011-07-22 15:32:56 UTC (rev 3326) @@ -250,10 +250,9 @@ <code>boolean</code> </entry> <entry> - <para>A change was made in JDBC4 so that when an 'Alias' is used it will - now be returned as the label. Prior to this, it was returned as - the name. Setting this property to false will enable - backwards compatibility when JDBC3 and older support is still required. + <para>A change was made in JDBC4 to return unaliased column names as the ResultSetMetadata column name. + Prior to this, if a column alias were used it was returned as the column name. Setting this property to false will enable + backwards compatibility when JDBC3 and older support is still required. Defaults to true. </para> </entry> </row> Modified: branches/7.4.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-extensions.xml =================================================================== --- branches/7.4.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-extensions.xml 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-extensions.xml 2011-07-22 15:32:56 UTC (rev 3326) @@ -175,11 +175,7 @@ <code>JDBC4COLUMNNAMEANDLABELSEMANTICS / useJDBC4ColumnNameAndLabelSemantics</code> </entry> <entry> - <para>A change was made in JDBC4 so that when an 'Alias' is used it will. - now be returned as the label. Prior to this, it was returned as - the name. Setting this property to false will enable - backwards compatibility when JDBC3 and older support is still required. - </para> + <para>Same as the connection property.</para> </entry> </row> </tbody> Modified: branches/7.4.x/runtime/src/main/java/org/teiid/odbc/ODBCServerRemoteImpl.java =================================================================== --- branches/7.4.x/runtime/src/main/java/org/teiid/odbc/ODBCServerRemoteImpl.java 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/runtime/src/main/java/org/teiid/odbc/ODBCServerRemoteImpl.java 2011-07-22 15:32:56 UTC (rev 3326) @@ -966,7 +966,7 @@ final ArrayList<PgColInfo> result = new ArrayList<PgColInfo>(columns); for (int i = 1; i < columns + 1; i++) { final PgColInfo info = new PgColInfo(); - info.name = meta.getColumnName(i).toLowerCase(); + info.name = meta.getColumnLabel(i).toLowerCase(); info.type = meta.getColumnType(i); info.type = convertType(info.type); info.precision = meta.getColumnDisplaySize(i); Modified: branches/7.4.x/test-integration/common/src/test/java/org/teiid/transport/TestODBCSocketTransport.java =================================================================== --- branches/7.4.x/test-integration/common/src/test/java/org/teiid/transport/TestODBCSocketTransport.java 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/test-integration/common/src/test/java/org/teiid/transport/TestODBCSocketTransport.java 2011-07-22 15:32:56 UTC (rev 3326) @@ -222,6 +222,12 @@ TestMMDatabaseMetaData.compareResultSet(rs); } + @Test public void testColumnMetadataWithAlias() throws Exception { + PreparedStatement stmt = conn.prepareStatement("select ta.attname as x from pg_catalog.pg_attribute ta limit 1"); + ResultSet rs = stmt.executeQuery(); + TestMMDatabaseMetaData.compareResultSet(rs); + } + @Test public void testPreparedError() throws Exception { PreparedStatement stmt = conn.prepareStatement("select cast(? as integer)"); stmt.setString(1, "a"); Added: branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testColumnMetadataWithAlias.expected =================================================================== --- branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testColumnMetadataWithAlias.expected (rev 0) +++ branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testColumnMetadataWithAlias.expected 2011-07-22 15:32:56 UTC (rev 3326) @@ -0,0 +1,6 @@ +varchar +x +PART_ID +Row Count : 1 +getColumnName getColumnType getCatalogName getColumnClassName getColumnLabel getColumnTypeName getSchemaName getTableName getColumnDisplaySize getPrecision getScale isAutoIncrement isCaseSensitive isCurrency isDefinitelyWritable isNullable isReadOnly isSearchable isSigned isWritable +x 12 java.lang.String x varchar 2147483647 0 0 false true false false 1 false true false true Modified: branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPk.expected =================================================================== --- branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPk.expected 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPk.expected 2011-07-22 15:32:56 UTC (rev 3326) @@ -1,10 +1,10 @@ varchar int2 varchar varchar varchar -name attnum tablename schemaname tablename +attname attnum relname nspname relname oid 1 pg_attribute pg_catalog pg_attribute Row Count : 1 getColumnName getColumnType getCatalogName getColumnClassName getColumnLabel getColumnTypeName getSchemaName getTableName getColumnDisplaySize getPrecision getScale isAutoIncrement isCaseSensitive isCurrency isDefinitelyWritable isNullable isReadOnly isSearchable isSigned isWritable -name 12 java.lang.String name varchar 2147483647 0 0 false true false false 0 false true false true +attname 12 java.lang.String attname varchar 2147483647 0 0 false true false false 0 false true false true attnum 5 java.lang.Integer attnum int2 6 5 0 false false false false 2 false true true true -tablename 12 java.lang.String tablename varchar 2147483647 0 0 false true false false 0 false true false true -schemaname 12 java.lang.String schemaname varchar 2147483647 0 0 false true false false 1 false true false true -tablename 12 java.lang.String tablename varchar 2147483647 0 0 false true false false 0 false true false true +relname 12 java.lang.String relname varchar 2147483647 0 0 false true false false 0 false true false true +nspname 12 java.lang.String nspname varchar 2147483647 0 0 false true false false 1 false true false true +relname 12 java.lang.String relname varchar 2147483647 0 0 false true false false 0 false true false true Modified: branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPkPrepared.expected =================================================================== --- branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPkPrepared.expected 2011-07-22 10:33:31 UTC (rev 3325) +++ branches/7.4.x/test-integration/common/src/test/resources/TestODBCSocketTransport/testPkPrepared.expected 2011-07-22 15:32:56 UTC (rev 3326) @@ -1,10 +1,10 @@ varchar int2 varchar varchar varchar -name attnum tablename schemaname tablename +attname attnum relname nspname relname oid 1 pg_attribute pg_catalog pg_attribute Row Count : 1 getColumnName getColumnType getCatalogName getColumnClassName getColumnLabel getColumnTypeName getSchemaName getTableName getColumnDisplaySize getPrecision getScale isAutoIncrement isCaseSensitive isCurrency isDefinitelyWritable isNullable isReadOnly isSearchable isSigned isWritable -name 12 java.lang.String name varchar 2147483647 0 0 false true false false 0 false true false true +attname 12 java.lang.String attname varchar 2147483647 0 0 false true false false 0 false true false true attnum 5 java.lang.Integer attnum int2 6 5 0 false false false false 2 false true true true -tablename 12 java.lang.String tablename varchar 2147483647 0 0 false true false false 0 false true false true -schemaname 12 java.lang.String schemaname varchar 2147483647 0 0 false true false false 1 false true false true -tablename 12 java.lang.String tablename varchar 2147483647 0 0 false true false false 0 false true false true +relname 12 java.lang.String relname varchar 2147483647 0 0 false true false false 0 false true false true +nspname 12 java.lang.String nspname varchar 2147483647 0 0 false true false false 1 false true false true +relname 12 java.lang.String relname varchar 2147483647 0 0 false true false false 0 false true false true [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3325 - in trunk/engine/src: test/java/org/teiid/query/optimizer and 1 other directory.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-22 06:33:31 -0400 (Fri, 22 Jul 2011) New Revision: 3325 Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java Log: TEIID-1665 optimizing grouping of all constants Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java ============================================================… [View More]======= --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java 2011-07-22 10:29:03 UTC (rev 3324) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java 2011-07-22 10:33:31 UTC (rev 3325) @@ -42,7 +42,9 @@ import org.teiid.query.optimizer.relational.RuleStack; import org.teiid.query.optimizer.relational.plantree.NodeConstants; import org.teiid.query.optimizer.relational.plantree.NodeEditor; +import org.teiid.query.optimizer.relational.plantree.NodeFactory; import org.teiid.query.optimizer.relational.plantree.PlanNode; +import org.teiid.query.optimizer.relational.plantree.NodeConstants.Info; import org.teiid.query.processor.relational.RelationalNode; import org.teiid.query.sql.lang.Command; import org.teiid.query.sql.lang.Criteria; @@ -51,6 +53,7 @@ import org.teiid.query.sql.lang.StoredProcedure; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; +import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.ExpressionSymbol; @@ -218,6 +221,30 @@ } List<SingleElementSymbol> requiredInput = collectRequiredInputSymbols(root); + //targeted optimization for unnecessary aggregation + if (root.getType() == NodeConstants.Types.GROUP && root.hasBooleanProperty(Info.IS_OPTIONAL) && NodeEditor.findParent(root, NodeConstants.Types.ACCESS) == null) { + PlanNode old = root; + PlanNode next = root.getFirstChild(); + NodeEditor.removeChildNode(root.getParent(), root); + + if (old.hasCollectionProperty(Info.GROUP_COLS)) { + SymbolMap symbolMap = (SymbolMap) old.getProperty(NodeConstants.Info.SYMBOL_MAP); + FrameUtil.convertFrame(next.getParent(), symbolMap.asMap().keySet().iterator().next().getGroupSymbol(), null, symbolMap.asMap(), metadata); + PlanNode limit = NodeFactory.getNewNode(NodeConstants.Types.TUPLE_LIMIT); + limit.setProperty(Info.MAX_TUPLE_LIMIT, new Constant(1)); + PlanNode parent = next.getParent(); + while (parent.getParent() != null && parent.getParent().getType() != NodeConstants.Types.SOURCE) { + parent = parent.getParent(); + } + if (!rules.contains(RuleConstants.PUSH_LIMIT)) { + rules.push(RuleConstants.PUSH_LIMIT); + } + parent.getFirstChild().addAsParent(limit); + execute(parent, metadata, capFinder, rules, analysisRecord, context); + return; + } + root = next; + } // Call children recursively if(root.getChildCount() == 1) { @@ -482,7 +509,7 @@ ElementCollectorVisitor.getElements(expression, requiredSymbols); } } - + boolean hasAggregate = false; SymbolMap symbolMap = (SymbolMap) node.getProperty(NodeConstants.Info.SYMBOL_MAP); // Take credit for creating any aggregates that are needed above @@ -491,7 +518,7 @@ Expression ex = symbolMap.getMappedExpression((ElementSymbol) outputSymbol); if(ex instanceof AggregateSymbol) { AggregateSymbol agg = (AggregateSymbol)ex; - + hasAggregate = true; Expression aggExpr = agg.getExpression(); if(aggExpr != null) { ElementCollectorVisitor.getElements(aggExpr, requiredSymbols); @@ -506,7 +533,9 @@ } } } - + if (requiredSymbols.isEmpty() && !hasAggregate) { + node.setProperty(Info.IS_OPTIONAL, true); + } break; } Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2011-07-22 10:29:03 UTC (rev 3324) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2011-07-22 10:33:31 UTC (rev 3325) @@ -6607,6 +6607,34 @@ helpPlan("select count(*) from agg3", metadata, new String[] {"SELECT COUNT(*) FROM (SELECT COUNT(*) AS c_0 FROM BQT1.SmallA AS g_0) AS v_0"}, new DefaultCapabilitiesFinder(bac), ComparisonMode.EXACT_COMMAND_STRING); } + + @Test public void testMergeGroupBy1() throws Exception { + BasicSourceCapabilities caps = new BasicSourceCapabilities(); + caps.setCapabilitySupport(Capability.ROW_LIMIT, true); + caps.setFunctionSupport("+", true); //$NON-NLS-1$ + ProcessorPlan plan = TestOptimizer.helpPlan("SELECT a, b FROM (select 1 as a, 2 as b from pm1.g1) as x group by a, b", //$NON-NLS-1$ + RealMetadataFactory.example1Cached(), null, new DefaultCapabilitiesFinder(caps), + new String[] { + "SELECT pm1.g1.e1 FROM pm1.g1 LIMIT 1"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + + checkNodeTypes(plan, new int[] { + 1, // Access + 0, // DependentAccess + 0, // DependentSelect + 0, // DependentProject + 0, // DupRemove + 0, // Grouping + 0, // NestedLoopJoinStrategy + 0, // MergeJoinStrategy + 0, // Null + 0, // PlanExecution + 1, // Project + 0, // Select + 0, // Sort + 0 // UnionAll + }); + } + public static final boolean DEBUG = false; } [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3324 - in branches/7.4.x/engine/src: test/java/org/teiid/query/rewriter and 1 other directory.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-22 06:29:03 -0400 (Fri, 22 Jul 2011) New Revision: 3324 Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java branches/7.4.x/engine/src/test/java/org/teiid/query/rewriter/TestQueryRewriter.java Log: TEIID-1645 re-enabling agg rewrite optimization Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java =================================================================== --- branches/7.4.x/… [View More]engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java 2011-07-21 19:28:44 UTC (rev 3323) +++ branches/7.4.x/engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java 2011-07-22 10:29:03 UTC (rev 3324) @@ -240,6 +240,7 @@ * @throws QueryValidatorException */ private Command rewriteCommand(Command command, boolean removeOrderBy) throws TeiidComponentException, TeiidProcessingException{ + boolean oldRewriteAggs = rewriteAggs; QueryMetadataInterface oldMetadata = metadata; CreateUpdateProcedureCommand oldProcCommand = procCommand; @@ -299,6 +300,7 @@ break; } + this.rewriteAggs = oldRewriteAggs; this.metadata = oldMetadata; this.procCommand = oldProcCommand; return command; @@ -764,6 +766,7 @@ */ private Query rewriteGroupBy(Query query) throws TeiidComponentException, TeiidProcessingException{ if (query.getGroupBy() == null) { + rewriteAggs = false; return query; } if (isDistinctWithGroupBy(query)) { @@ -1664,6 +1667,7 @@ private BigDecimal BIG_DECIMAL_ZERO = new BigDecimal("0"); //$NON-NLS-1$ private Short SHORT_ZERO = new Short((short)0); private Byte BYTE_ZERO = new Byte((byte)0); + private boolean rewriteAggs = true; /** * @param criteria @@ -2358,6 +2362,9 @@ expression.setAggregateFunction(Type.MAX); } } + if (rewriteAggs && expression.getExpression() != null && EvaluatableVisitor.willBecomeConstant(expression.getExpression())) { + return expression.getExpression(); + } return expression; } Modified: branches/7.4.x/engine/src/test/java/org/teiid/query/rewriter/TestQueryRewriter.java =================================================================== --- branches/7.4.x/engine/src/test/java/org/teiid/query/rewriter/TestQueryRewriter.java 2011-07-21 19:28:44 UTC (rev 3323) +++ branches/7.4.x/engine/src/test/java/org/teiid/query/rewriter/TestQueryRewriter.java 2011-07-22 10:29:03 UTC (rev 3324) @@ -2460,4 +2460,10 @@ @Test public void testRewriteNestedConvert2() throws Exception { helpTestRewriteExpression("cast(cast(pm1.g1.e3 as string) as clob)", "convert(convert(pm1.g1.e3, string), clob)", RealMetadataFactory.example1Cached()); //$NON-NLS-1$ //$NON-NLS-2$ } + + @Test public void testRewriteConstantAgg() throws Exception { + helpTestRewriteCommand("select max(1) from pm1.g1 group by e1", "SELECT 1 FROM pm1.g1 GROUP BY e1"); + } + + } [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3323 - in trunk: documentation/developer-guide/src/main/docbook/en-US/content and 22 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-21 15:28:44 -0400 (Thu, 21 Jul 2011) New Revision: 3323 Removed: trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllInGroupSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllSymbol.java trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java Modified: trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-… [View More]api.xml trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java trunk/engine/src/main/java/org/teiid/query/optimizer/xml/QueryUtil.java trunk/engine/src/main/java/org/teiid/query/optimizer/xml/XMLPlanner.java trunk/engine/src/main/java/org/teiid/query/resolver/command/SimpleQueryResolver.java trunk/engine/src/main/java/org/teiid/query/resolver/command/XMLQueryResolver.java trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/lang/Select.java trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/DerivedColumn.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/MultipleElementSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/SelectSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/SingleElementSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/AbstractSymbolMappingVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj trunk/engine/src/main/resources/org/teiid/query/i18n.properties trunk/engine/src/test/java/org/teiid/cdk/CommandBuilder.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java trunk/engine/src/test/java/org/teiid/query/optimizer/relational/rules/TestRulePlaceAccess.java trunk/engine/src/test/java/org/teiid/query/parser/TestLimitParsing.java trunk/engine/src/test/java/org/teiid/query/parser/TestOptionsAndHints.java trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java trunk/engine/src/test/java/org/teiid/query/parser/TestSetQueryParsing.java trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java trunk/engine/src/test/java/org/teiid/query/sql/lang/TestSelect.java trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestAggregateSymbolCollectorVisitor.java trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestSQLStringVisitor.java trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestStaticSymbolMappingVisitor.java trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java Log: TEIID-1667 refining agg and window function validation and also continuing to refine the language object model to move beyond the symbol base class Modified: trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java =================================================================== --- trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -725,8 +725,6 @@ /** * Support indicates that the connector supports functions in GROUP BY, such as: * <code>SELECT dayofmonth(theDate), COUNT(*) FROM table GROUP BY dayofmonth(theDate)</code> - * - * NOT CURRENTLY USED - group by expressions create an inline view for pushdown * @since 5.0 */ public boolean supportsFunctionsInGroupBy() { Modified: trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-api.xml =================================================================== --- trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-api.xml 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/documentation/developer-guide/src/main/docbook/en-US/content/translator-api.xml 2011-07-21 19:28:44 UTC (rev 3323) @@ -1229,7 +1229,7 @@ <para>GroupBy</para> </entry> <entry> - <para>Not currently used - non-element expressions in the group by create an inline view.</para> + <para>Translator can support non-column reference grouping expressions.</para> </entry> </row> <row> @@ -1287,6 +1287,28 @@ <para>Translator supports the WITH clause.</para> </entry> </row> + <row> + <entry> + <para>ElementaryOlapOperations</para> + </entry> + <entry> + <para/> + </entry> + <entry> + <para>Translator supports window functions and analytic functions RANK, DENSE_RANK, and ROW_NUMBER.</para> + </entry> + </row> + <row> + <entry> + <para>AdvancedOlapOperations</para> + </entry> + <entry> + <para/> + </entry> + <entry> + <para>Translator supports aggregate conditions.</para> + </entry> + </row> </tbody> </tgroup> </table> Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/MetaDataProcessor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -180,12 +180,12 @@ this.metadata = tempFacade; } - List projectedSymbols = originalCommand.getProjectedSymbols(); + List<SingleElementSymbol> projectedSymbols = originalCommand.getProjectedSymbols(); columnMetadata = new Map[projectedSymbols.size()]; - Iterator symbolIter = projectedSymbols.iterator(); + Iterator<SingleElementSymbol> symbolIter = projectedSymbols.iterator(); for(int i=0; symbolIter.hasNext(); i++) { - SingleElementSymbol symbol = (SingleElementSymbol) symbolIter.next(); + SingleElementSymbol symbol = symbolIter.next(); String shortColumnName = SingleElementSymbol.getShortName(symbol.getOutputName()); if(symbol instanceof AliasSymbol) { symbol = ((AliasSymbol)symbol).getSymbol(); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -112,7 +112,7 @@ import org.teiid.query.sql.proc.TriggerAction; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; @@ -744,7 +744,7 @@ Collection<AggregateSymbol> aggs = AggregateSymbolCollectorVisitor.getAggregates(query.getSelect(), true); boolean hasGrouping = !aggs.isEmpty(); if (query.getHaving() != null) { - AggregateSymbolCollectorVisitor.getAggregates(query.getHaving(), aggs, null); + aggs.addAll(AggregateSymbolCollectorVisitor.getAggregates(query.getHaving(), true)); hasGrouping = true; } if (query.getGroupBy() != null) { @@ -1224,7 +1224,7 @@ }else{ this.context.accessedPlanningObject(matMetadataId); qnode = new QueryNode(null); - Query query = createMatViewQuery(matMetadataId, matTableName, Arrays.asList(new AllSymbol()), isImplicitGlobal); + Query query = createMatViewQuery(matMetadataId, matTableName, Arrays.asList(new MultipleElementSymbol()), isImplicitGlobal); query.setCacheHint(hint); qnode.setCommand(query); cacheString = "matview"; //$NON-NLS-1$ Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -24,11 +24,9 @@ import java.util.ArrayList; import java.util.Collection; -import java.util.HashMap; import java.util.Iterator; import java.util.List; import java.util.Set; -import java.util.concurrent.atomic.AtomicInteger; import org.teiid.api.exception.query.QueryMetadataException; import org.teiid.core.TeiidComponentException; @@ -169,22 +167,14 @@ } } - private static AtomicInteger EXPRESSION_INDEX = new AtomicInteger(0); - public static List<SingleElementSymbol> createExpressionSymbols(List<? extends Expression> expressions) { - HashMap<Expression, ExpressionSymbol> uniqueExpressions = new HashMap<Expression, ExpressionSymbol>(); List<SingleElementSymbol> result = new ArrayList<SingleElementSymbol>(); for (Expression expression : expressions) { if (expression instanceof SingleElementSymbol) { result.add((SingleElementSymbol)expression); continue; } - ExpressionSymbol expressionSymbol = uniqueExpressions.get(expression); - if (expressionSymbol == null) { - expressionSymbol = new ExpressionSymbol("$" + EXPRESSION_INDEX.getAndIncrement(), expression); //$NON-NLS-1$ - uniqueExpressions.put(expression, expressionSymbol); - } - result.add(expressionSymbol); + result.add(new ExpressionSymbol("expr", expression)); //$NON-NLS-1$ } return result; } Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -68,10 +68,10 @@ import org.teiid.query.sql.lang.SubquerySetCriteria; import org.teiid.query.sql.navigator.DeepPostOrderNavigator; import org.teiid.query.sql.symbol.AggregateSymbol; -import org.teiid.query.sql.symbol.AliasSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; +import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; @@ -455,7 +455,7 @@ private boolean isSimpleJoin(Query query) { if (query.getFrom() != null) { - for (FromClause clause : (List<FromClause>)query.getFrom().getClauses()) { + for (FromClause clause : query.getFrom().getClauses()) { if (RuleCollapseSource.hasOuterJoins(clause)) { return false; } @@ -479,7 +479,7 @@ plannedResult.query.setLimit(null); List<GroupSymbol> rightGroups = plannedResult.query.getFrom().getGroups(); - Set<SingleElementSymbol> requiredExpressions = new LinkedHashSet<SingleElementSymbol>(); + Set<Expression> requiredExpressions = new LinkedHashSet<Expression>(); final SymbolMap refs = plannedResult.query.getCorrelatedReferences(); boolean addGroupBy = false; if (refs != null) { @@ -494,7 +494,7 @@ return false; } if (plannedResult.query.getGroupBy() == null) { - processCriteria(leftGroups, plannedResult, rightGroups, requiredExpressions, refs, where, true); + processCriteria(leftGroups, plannedResult, rightGroups, requiredExpressions, refs, where, null, true); if (hasAggregates) { if (!plannedResult.nonEquiJoinCriteria.isEmpty()) { return false; @@ -502,7 +502,7 @@ addGroupBy = true; } } - processCriteria(leftGroups, plannedResult, rightGroups, requiredExpressions, refs, having, false); + processCriteria(leftGroups, plannedResult, rightGroups, requiredExpressions, refs, having, plannedResult.query.getGroupBy(), false); } if (plannedResult.additionalCritieria != null) { @@ -525,29 +525,29 @@ } if (addGroupBy) { - LinkedHashSet<SingleElementSymbol> groupingSymbols = new LinkedHashSet<SingleElementSymbol>(); - ArrayList<SingleElementSymbol> aggs = new ArrayList<SingleElementSymbol>(); + LinkedHashSet<Expression> groupingSymbols = new LinkedHashSet<Expression>(); for (Expression expr : (List<Expression>)plannedResult.rightExpressions) { - AggregateSymbolCollectorVisitor.getAggregates(expr, aggs, groupingSymbols); + AggregateSymbolCollectorVisitor.getAggregates(expr, null, groupingSymbols, null, null, null); } if (!groupingSymbols.isEmpty()) { - plannedResult.query.setGroupBy((GroupBy) new GroupBy(new ArrayList<SingleElementSymbol>(groupingSymbols)).clone()); + plannedResult.query.setGroupBy((GroupBy) new GroupBy(new ArrayList<Expression>(groupingSymbols)).clone()); } } - HashSet<SingleElementSymbol> projectedSymbols = new HashSet<SingleElementSymbol>(); + HashSet<Expression> projectedSymbols = new HashSet<Expression>(); for (SingleElementSymbol ses : plannedResult.query.getProjectedSymbols()) { - if (ses instanceof AliasSymbol) { - ses = ((AliasSymbol)ses).getSymbol(); - } - projectedSymbols.add(ses); + projectedSymbols.add(SymbolMap.getExpression(ses)); } - for (SingleElementSymbol ses : requiredExpressions) { + for (Expression ses : requiredExpressions) { if (projectedSymbols.add(ses)) { - plannedResult.query.getSelect().addSymbol((SingleElementSymbol) ses.clone()); + if (ses instanceof SingleElementSymbol) { + plannedResult.query.getSelect().addSymbol((SingleElementSymbol)ses); + } else { + plannedResult.query.getSelect().addSymbol(new ExpressionSymbol("expr", (Expression) ses.clone())); //$NON-NLS-1$ + } } } for (SingleElementSymbol ses : (List<SingleElementSymbol>)plannedResult.rightExpressions) { - if (projectedSymbols.add(ses)) { + if (projectedSymbols.add(SymbolMap.getExpression(ses))) { plannedResult.query.getSelect().addSymbol((SingleElementSymbol)ses.clone()); } } @@ -556,8 +556,8 @@ private void processCriteria(Collection<GroupSymbol> leftGroups, PlannedResult plannedResult, List<GroupSymbol> rightGroups, - Set<SingleElementSymbol> requiredExpressions, final SymbolMap refs, - Criteria joinCriteria, boolean where) { + Set<Expression> requiredExpressions, final SymbolMap refs, + Criteria joinCriteria, GroupBy groupBy, boolean where) { if (joinCriteria == null) { return; } @@ -565,9 +565,8 @@ for (Iterator<Criteria> critIter = crits.iterator(); critIter.hasNext();) { Criteria conjunct = critIter.next(); - List<SingleElementSymbol> aggregates = new LinkedList<SingleElementSymbol>(); - List<SingleElementSymbol> elements = new LinkedList<SingleElementSymbol>(); - AggregateSymbolCollectorVisitor.getAggregates(conjunct, aggregates, elements); + List<Expression> additionalRequired = new LinkedList<Expression>(); + AggregateSymbolCollectorVisitor.getAggregates(conjunct, additionalRequired, additionalRequired, additionalRequired, null, groupBy!=null?groupBy.getSymbols():null); ReferenceReplacementVisitor emv = new ReferenceReplacementVisitor(refs); DeepPostOrderNavigator.doVisit(conjunct, emv); if (!emv.replacedAny) { @@ -579,8 +578,7 @@ plannedResult.query.setHaving(Criteria.combineCriteria(plannedResult.query.getHaving(), conjunct)); } } else { - requiredExpressions.addAll(aggregates); - requiredExpressions.addAll(elements); + requiredExpressions.addAll(additionalRequired); } } RuleChooseJoinStrategy.separateCriteria(leftGroups, rightGroups, plannedResult.leftExpressions, plannedResult.rightExpressions, crits, plannedResult.nonEquiJoinCriteria); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -277,7 +277,7 @@ if (ex instanceof AggregateSymbol) { compositeAggs.add((AggregateSymbol) ex); } else { - AggregateSymbolCollectorVisitor.getAggregates(ex, compositeAggs, null); + compositeAggs.addAll(AggregateSymbolCollectorVisitor.getAggregates(ex, false)); hasExpressionMapping = true; } } Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/xml/QueryUtil.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/xml/QueryUtil.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/xml/QueryUtil.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -51,10 +51,10 @@ import org.teiid.query.sql.lang.Query; import org.teiid.query.sql.lang.Select; import org.teiid.query.sql.navigator.DeepPreOrderNavigator; -import org.teiid.query.sql.symbol.AllInGroupSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.visitor.ElementCollectorVisitor; import org.teiid.query.sql.visitor.ReferenceCollectorVisitor; @@ -135,7 +135,7 @@ static Query wrapQuery(FromClause fromClause, String groupName) { Select select = new Select(); - select.addSymbol(new AllInGroupSymbol(groupName + ".*")); //$NON-NLS-1$ + select.addSymbol(new MultipleElementSymbol(groupName)); Query query = new Query(); query.setSelect(select); From from = new From(); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/xml/XMLPlanner.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/xml/XMLPlanner.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/xml/XMLPlanner.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -53,9 +53,9 @@ import org.teiid.query.sql.lang.OrderBy; import org.teiid.query.sql.lang.Query; import org.teiid.query.sql.lang.Select; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.SelectSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.visitor.ElementCollectorVisitor; @@ -247,7 +247,7 @@ SelectSymbol firstSymbol = select.getSymbol(0); // 0. mark the nodes to be excluded - if(firstSymbol instanceof AllSymbol) { + if(firstSymbol instanceof MultipleElementSymbol && ((MultipleElementSymbol)firstSymbol).getGroup() == null) { return doc; } Modified: trunk/engine/src/main/java/org/teiid/query/resolver/command/SimpleQueryResolver.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/resolver/command/SimpleQueryResolver.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/resolver/command/SimpleQueryResolver.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -76,12 +76,11 @@ import org.teiid.query.sql.lang.XMLTable; import org.teiid.query.sql.navigator.PostOrderNavigator; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SingleElementSymbol; @@ -89,8 +88,6 @@ public class SimpleQueryResolver implements CommandResolver { - private static final String ALL_IN_GROUP_SUFFIX = ".*"; //$NON-NLS-1$ - /** * @see org.teiid.query.resolver.CommandResolver#resolveCommand(org.teiid.query.sql.lang.Command, org.teiid.query.metadata.TempMetadataAdapter, boolean) */ @@ -107,14 +104,14 @@ ResolverVisitor visitor = (ResolverVisitor)qrv.getVisitor(); visitor.throwException(true); } catch (TeiidRuntimeException e) { - if (e.getChild() instanceof QueryMetadataException) { - throw (QueryMetadataException)e.getChild(); + if (e.getCause() instanceof QueryMetadataException) { + throw (QueryMetadataException)e.getCause(); } - if (e.getChild() instanceof QueryResolverException) { - throw (QueryResolverException)e.getChild(); + if (e.getCause() instanceof QueryResolverException) { + throw (QueryResolverException)e.getCause(); } - if (e.getChild() instanceof TeiidComponentException) { - throw (TeiidComponentException)e.getChild(); + if (e.getCause() instanceof TeiidComponentException) { + throw (TeiidComponentException)e.getCause(); } throw e; } @@ -185,18 +182,18 @@ } } - private static GroupSymbol resolveAllInGroup(AllInGroupSymbol allInGroupSymbol, Set<GroupSymbol> groups, QueryMetadataInterface metadata) throws QueryResolverException, QueryMetadataException, TeiidComponentException { - String name = allInGroupSymbol.getName(); - int index = name.lastIndexOf(ALL_IN_GROUP_SUFFIX); - String groupAlias = name.substring(0, index); - List<GroupSymbol> groupSymbols = ResolverUtil.findMatchingGroups(groupAlias.toUpperCase(), groups, metadata); + private static GroupSymbol resolveAllInGroup(MultipleElementSymbol allInGroupSymbol, Set<GroupSymbol> groups, QueryMetadataInterface metadata) throws QueryResolverException, QueryMetadataException, TeiidComponentException { + String groupAlias = allInGroupSymbol.getGroup().getCanonicalName(); + List<GroupSymbol> groupSymbols = ResolverUtil.findMatchingGroups(groupAlias, groups, metadata); if(groupSymbols.isEmpty() || groupSymbols.size() > 1) { String msg = QueryPlugin.Util.getString(groupSymbols.isEmpty()?"ERR.015.008.0047":"SimpleQueryResolver.ambiguous_all_in_group", allInGroupSymbol); //$NON-NLS-1$ //$NON-NLS-2$ QueryResolverException qre = new QueryResolverException(msg); qre.addUnresolvedSymbol(new UnresolvedSymbolDescription(allInGroupSymbol.toString(), msg)); throw qre; } - + GroupSymbol gs = allInGroupSymbol.getGroup(); + allInGroupSymbol.setGroup(groupSymbols.get(0).clone()); + allInGroupSymbol.getGroup().setOutputName(gs.getOutputName()); return groupSymbols.get(0); } @@ -260,14 +257,19 @@ } } - public void visit(AllSymbol obj) { + public void visit(MultipleElementSymbol obj) { + // Determine group that this symbol is for try { List<ElementSymbol> elementSymbols = new ArrayList<ElementSymbol>(); - for (GroupSymbol group : currentGroups) { + Collection<GroupSymbol> groups = currentGroups; + if (obj.getGroup() != null) { + groups = Arrays.asList(resolveAllInGroup(obj, currentGroups, metadata)); + } + for (GroupSymbol group : groups) { elementSymbols.addAll(resolveSelectableElements(group)); } obj.setElementSymbols(elementSymbols); - } catch (TeiidComponentException err) { + } catch (TeiidException err) { throw new TeiidRuntimeException(err); } } @@ -289,19 +291,6 @@ return result; } - public void visit(AllInGroupSymbol obj) { - // Determine group that this symbol is for - try { - GroupSymbol group = resolveAllInGroup(obj, currentGroups, metadata); - - List<ElementSymbol> elements = resolveSelectableElements(group); - - obj.setElementSymbols(elements); - } catch (TeiidException err) { - throw new TeiidRuntimeException(err); - } - } - public void visit(ScalarSubquery obj) { resolveSubQuery(obj, this.currentGroups); } @@ -471,7 +460,7 @@ from.addClause(new SubqueryFromClause("X", storedProcedureCommand)); //$NON-NLS-1$ procQuery.setFrom(from); Select select = new Select(); - select.addSymbol(new AllInGroupSymbol("X.*")); //$NON-NLS-1$ + select.addSymbol(new MultipleElementSymbol("X")); //$NON-NLS-1$ procQuery.setSelect(select); List<String> accessPatternElementNames = new LinkedList<String>(); Modified: trunk/engine/src/main/java/org/teiid/query/resolver/command/XMLQueryResolver.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/resolver/command/XMLQueryResolver.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/resolver/command/XMLQueryResolver.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -58,11 +58,10 @@ import org.teiid.query.sql.lang.Select; import org.teiid.query.sql.lang.SubqueryContainer; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.SelectSymbol; import org.teiid.query.sql.visitor.ElementCollectorVisitor; import org.teiid.query.sql.visitor.GroupCollectorVisitor; @@ -337,47 +336,38 @@ // There are other cases of "xml", such as, element name = "xml", // but those are ok because those will be resolved later as normal elements - String symbolName = ss.getName(); + ElementSymbol es = (ElementSymbol)ss; + String symbolName = es.getName(); if(!subquery && (symbolName.equalsIgnoreCase("xml") || symbolName.equalsIgnoreCase(group.getName() + ".xml"))) { //$NON-NLS-1$ //$NON-NLS-2$ if(elements.size() != 1) { throw new QueryResolverException(QueryPlugin.Util.getString("XMLQueryResolver.xml_only_valid_alone")); //$NON-NLS-1$ } select.clearSymbols(); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); all.setElementSymbols(validElements.values()); select.addSymbol(all); query.setSelect(select); return; } // normal elements - resolveElement((ElementSymbol)ss, validElements, externalGroups, metadata); - } else if (ss instanceof AllInGroupSymbol) { + resolveElement(es, validElements, externalGroups, metadata); + } else if (ss instanceof MultipleElementSymbol) { // Resolve the element with "*" case. such as "A.*" // by stripping off the ".*" part, - String symbolName = ss.getName(); - int index = symbolName.indexOf("*"); //$NON-NLS-1$ - String elementPart = symbolName.substring(0, index-1); + MultipleElementSymbol all = (MultipleElementSymbol)ss; // Check for case where we have model.doc.* - if(elementPart.equalsIgnoreCase(group.getName())) { - select.clearSymbols(); - AllSymbol all = new AllSymbol(); + if(all.getGroup() == null || all.getGroup().getName().equalsIgnoreCase(group.getName())) { all.setElementSymbols(validElements.values()); - select.addSymbol(all); - query.setSelect(select); - } else { - // resovlve the node which is specified - ElementSymbol elementSymbol = new ElementSymbol(elementPart); - resolveElement(elementSymbol, validElements, externalGroups, metadata); - - // now find all the elements under this node and set as elements. - List<ElementSymbol> elementsInNode = getElementsUnderNode(elementSymbol.getMetadataID(), validElements.values(), metadata); - ((AllInGroupSymbol)ss).setElementSymbols(elementsInNode); + return; } - } else if (ss instanceof AllSymbol) { - AllSymbol all = (AllSymbol)ss; - all.setElementSymbols(validElements.values()); - return; + // resovlve the node which is specified + ElementSymbol elementSymbol = new ElementSymbol(all.getGroup().getName()); + resolveElement(elementSymbol, validElements, externalGroups, metadata); + + // now find all the elements under this node and set as elements. + List<ElementSymbol> elementsInNode = getElementsUnderNode(elementSymbol.getMetadataID(), validElements.values(), metadata); + all.setElementSymbols(elementsInNode); } else if (ss instanceof ExpressionSymbol) { throw new QueryResolverException(QueryPlugin.Util.getString("XMLQueryResolver.no_expressions_in_select")); //$NON-NLS-1$ } else if (ss instanceof AliasSymbol) { Modified: trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -83,8 +83,6 @@ import org.teiid.query.sql.proc.WhileStatement; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.CaseExpression; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.DerivedColumn; @@ -92,6 +90,7 @@ import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.QueryString; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; @@ -172,8 +171,7 @@ // Visitor methods for symbol objects public void visit(AggregateSymbol obj) {} public void visit(AliasSymbol obj) {} - public void visit(AllInGroupSymbol obj) {} - public void visit(AllSymbol obj) {} + public void visit(MultipleElementSymbol obj) {} public void visit(Constant obj) {} public void visit(ElementSymbol obj) {} public void visit(ExpressionSymbol obj) {} Modified: trunk/engine/src/main/java/org/teiid/query/sql/lang/Select.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/lang/Select.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/lang/Select.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -30,7 +30,6 @@ import org.teiid.core.util.HashCodeUtil; import org.teiid.query.sql.LanguageObject; import org.teiid.query.sql.LanguageVisitor; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.SelectSymbol; @@ -85,7 +84,7 @@ * @return True if Select * is used */ public boolean isStar() { - return (symbols.size() == 1 && symbols.get(0) instanceof AllSymbol); + return (symbols.size() == 1 && symbols.get(0) instanceof MultipleElementSymbol && ((MultipleElementSymbol)symbols.get(0)).getGroup() == null); } /** Modified: trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -87,8 +87,6 @@ import org.teiid.query.sql.proc.WhileStatement; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.CaseExpression; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.DerivedColumn; @@ -97,6 +95,7 @@ import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.QueryString; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; @@ -154,14 +153,10 @@ visitNode(obj.getSymbol()); postVisitVisitor(obj); } - public void visit(AllInGroupSymbol obj) { + public void visit(MultipleElementSymbol obj) { preVisitVisitor(obj); postVisitVisitor(obj); } - public void visit(AllSymbol obj) { - preVisitVisitor(obj); - postVisitVisitor(obj); - } public void visit(AssignmentStatement obj) { preVisitVisitor(obj); visitNode(obj.getVariable()); Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -72,6 +72,7 @@ private boolean distinct; private OrderBy orderBy; private Expression condition; + private boolean isWindowed; private static final Class<Integer> COUNT_TYPE = DataTypeManager.DefaultDataClasses.INTEGER; private static final Map<Class<?>, Class<?>> SUM_TYPES; @@ -224,6 +225,7 @@ if (condition != null) { copy.setCondition((Expression) condition.clone()); } + copy.isWindowed = this.isWindowed; return copy; } @@ -247,6 +249,7 @@ return this.aggregate.equals(other.aggregate) && this.distinct == other.distinct + && this.isWindowed == other.isWindowed && EquivalenceUtil.areEqual(this.getExpression(), other.getExpression()) && EquivalenceUtil.areEqual(this.condition, other.condition) && EquivalenceUtil.areEqual(this.getOrderBy(), other.getOrderBy()); @@ -302,5 +305,13 @@ } return true; } + + public boolean isWindowed() { + return isWindowed; + } + + public void setWindowed(boolean isWindowed) { + this.isWindowed = isWindowed; + } } Deleted: trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllInGroupSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllInGroupSymbol.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllInGroupSymbol.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -1,72 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.query.sql.symbol; - -import java.util.List; - -import org.teiid.query.sql.LanguageObject; -import org.teiid.query.sql.LanguageVisitor; - -/** - * This is a subclass of Symbol representing <group>.*, which contains all of - * the elements from a single group in the query. The name of this symbol is "group.*", - * when resolved it contains a single Group Symbols and a set of Element Symbols - */ -public class AllInGroupSymbol extends MultipleElementSymbol { - - /** - * Constructor used for cloning - * @param name - * @param canonicalName - * @since 4.3 - */ - protected AllInGroupSymbol(String name, String canonicalName) { - super(name, canonicalName); - } - - /** - * Default constructor - */ - public AllInGroupSymbol(String name) { - super(name); - } - - public void acceptVisitor(LanguageVisitor visitor) { - visitor.visit(this); - } - - /** - * Return a deep copy of this object - * @return Deep copy of this object - */ - public Object clone() { - AllInGroupSymbol copy = new AllInGroupSymbol(getName(), getCanonical()); - - List<ElementSymbol> elements = getElementSymbols(); - if(elements != null && elements.size() > 0) { - copy.setElementSymbols(LanguageObject.Util.deepClone(elements, ElementSymbol.class)); - } - - return copy; - } -} Deleted: trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllSymbol.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/AllSymbol.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -1,75 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.query.sql.symbol; - -import java.util.List; - -import org.teiid.query.sql.LanguageObject; -import org.teiid.query.sql.LanguageVisitor; - -/** - * This is a subclass of Symbol representing *, which contains all of - * the elements from all groups in the query. The name of this symbol is always "*", - * when resolved it contains a set of Element Symbols referred to by the AllSymbol - */ -public class AllSymbol extends MultipleElementSymbol { - - private static final String ALL_SYMBOL_NAME = "*"; //$NON-NLS-1$ - - /** - * Constructor used for cloning - * @param name - * @param canonicalName - * @since 4.3 - */ - protected AllSymbol(String name, String canonicalName) { - super(name, canonicalName); - } - - /** - * Default constructor - */ - public AllSymbol(){ - this(ALL_SYMBOL_NAME, ALL_SYMBOL_NAME); - } - - public void acceptVisitor(LanguageVisitor visitor) { - visitor.visit(this); - } - - /** - * Return a deep copy of this object - * @return Deep copy of this object - */ - public Object clone() { - AllSymbol copy = new AllSymbol(ALL_SYMBOL_NAME, ALL_SYMBOL_NAME); - - List<ElementSymbol> elements = getElementSymbols(); - if(elements != null && elements.size() > 0) { - copy.setElementSymbols(LanguageObject.Util.deepClone(elements, ElementSymbol.class)); - } - - return copy; - } - -} Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/DerivedColumn.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/DerivedColumn.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/DerivedColumn.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -24,7 +24,6 @@ import org.teiid.core.util.EquivalenceUtil; import org.teiid.core.util.HashCodeUtil; -import org.teiid.query.function.source.XMLSystemFunctions; import org.teiid.query.sql.LanguageObject; import org.teiid.query.sql.LanguageVisitor; import org.teiid.query.sql.visitor.SQLStringVisitor; Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/MultipleElementSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/MultipleElementSymbol.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/MultipleElementSymbol.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -22,22 +22,23 @@ package org.teiid.query.sql.symbol; -import java.util.*; +import java.util.LinkedList; +import java.util.List; +import org.teiid.core.util.EquivalenceUtil; +import org.teiid.core.util.HashCodeUtil; +import org.teiid.query.sql.LanguageObject; +import org.teiid.query.sql.LanguageVisitor; +import org.teiid.query.sql.visitor.SQLStringVisitor; + /** * This is a subclass of Symbol representing multiple output columns. */ -public abstract class MultipleElementSymbol extends SelectSymbol { +public class MultipleElementSymbol implements SelectSymbol { private List<ElementSymbol> elementSymbols; + private GroupSymbol group; - /** - * Passthrough constructor used for cloning - * @param name - * @param canonicalName - * @since 4.3 - */ - protected MultipleElementSymbol(String name, String canonicalName) { - super(name, canonicalName); + public MultipleElementSymbol() { } /** @@ -45,7 +46,7 @@ * @param name Name of the symbol */ public MultipleElementSymbol(String name){ - super(name); + this.group = new GroupSymbol(name); } /** @@ -83,5 +84,60 @@ public boolean isResolved() { return(elementSymbols != null); } + + public void acceptVisitor(LanguageVisitor visitor) { + visitor.visit(this); + } + /** + * Return a deep copy of this object + * @return Deep copy of this object + */ + public Object clone() { + MultipleElementSymbol copy = new MultipleElementSymbol(); + if (group != null) { + copy.group = group.clone(); + } + + List<ElementSymbol> elements = getElementSymbols(); + if(elements != null && elements.size() > 0) { + copy.setElementSymbols(LanguageObject.Util.deepClone(elements, ElementSymbol.class)); + } + + return copy; + } + + /** + * @return null if selecting all groups, otherwise the specific group + */ + public GroupSymbol getGroup() { + return group; + } + + public void setGroup(GroupSymbol group) { + this.group = group; + } + + @Override + public String toString() { + return SQLStringVisitor.getSQLString(this); + } + + @Override + public int hashCode() { + return HashCodeUtil.hashCode(0, group); + } + + @Override + public boolean equals(Object obj) { + if (obj == this) { + return true; + } + if (!(obj instanceof MultipleElementSymbol)) { + return false; + } + MultipleElementSymbol other = (MultipleElementSymbol)obj; + return EquivalenceUtil.areEqual(this.group, other.group); + } + } Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/SelectSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/SelectSymbol.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/SelectSymbol.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -22,29 +22,14 @@ package org.teiid.query.sql.symbol; +import org.teiid.query.sql.LanguageObject; + /** * This is the server's representation of a metadata symbol that can be used * in a SELECT statement. It exists as a typing mechanism and to provide a single * additional method to get an ordered list of elements from each symbol. */ -public abstract class SelectSymbol extends Symbol { +public interface SelectSymbol extends LanguageObject { + - /** - * Passthrough constructor used for cloning - * @param name - * @param canonicalName - * @since 4.3 - */ - protected SelectSymbol(String name, String canonicalName) { - super(name, canonicalName); - } - - /** - * Construct a symbol with a name - * @param name Name of symbol - */ - public SelectSymbol(String name) { - super(name); - } - } Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/SingleElementSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/SingleElementSymbol.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/SingleElementSymbol.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -27,7 +27,7 @@ /** * This is a subclass of Symbol representing a single output column. */ -public abstract class SingleElementSymbol extends SelectSymbol implements Expression { +public abstract class SingleElementSymbol extends Symbol implements Expression, SelectSymbol { /** * Character used to delimit name components in a symbol Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -29,6 +29,7 @@ import org.teiid.query.sql.LanguageObject; import org.teiid.query.sql.LanguageVisitor; import org.teiid.query.sql.lang.OrderBy; +import org.teiid.query.sql.visitor.SQLStringVisitor; public class WindowFunction implements Expression { @@ -46,6 +47,7 @@ public void setFunction(AggregateSymbol expression) { this.function = expression; + this.function.setWindowed(true); } public List<Expression> getPartition() { @@ -111,4 +113,9 @@ return clone; } + @Override + public String toString() { + return SQLStringVisitor.getSQLString(this); + } + } Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/AbstractSymbolMappingVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/AbstractSymbolMappingVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/AbstractSymbolMappingVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -33,11 +33,10 @@ import org.teiid.query.sql.lang.UnaryFromClause; import org.teiid.query.sql.lang.Update; import org.teiid.query.sql.proc.CriteriaSelector; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Symbol; @@ -124,70 +123,28 @@ * Swap each ElementSymbol referenced by AllInGroupSymbol * @param obj Object to remap */ - public void visit(AllInGroupSymbol obj) { - // Discover new group name during course of mapping - String newGroupName = null; - - List oldSymbols = obj.getElementSymbols(); + public void visit(MultipleElementSymbol obj) { + List<ElementSymbol> oldSymbols = obj.getElementSymbols(); if(oldSymbols != null && oldSymbols.size() > 0) { - List newSymbols = new ArrayList(oldSymbols.size()); + List<ElementSymbol> newSymbols = new ArrayList<ElementSymbol>(oldSymbols.size()); - Iterator iter = oldSymbols.iterator(); + Iterator<ElementSymbol> iter = oldSymbols.iterator(); while(iter.hasNext()) { - ElementSymbol es = (ElementSymbol) iter.next(); + ElementSymbol es = iter.next(); ElementSymbol mappedSymbol = getMappedElement(es); - - // Save group name on first valid mapped element - if(newGroupName == null && mappedSymbol != null) { - GroupSymbol newGroup = mappedSymbol.getGroupSymbol(); - if(newGroup != null) { - newGroupName = newGroup.getName(); - } - } - newSymbols.add( mappedSymbol ); } obj.setElementSymbols(newSymbols); } - - // If haven't discovered group name yet (if, for instance, stuff isn't resolved), - // then fake up a group symbol, map it, and use the name of the mapped group symbol - if(newGroupName == null) { - String symbolName = obj.getName(); - String oldGroupName = symbolName.substring(0, symbolName.length()-2); // cut .* off - - GroupSymbol fakeSymbol = new GroupSymbol(oldGroupName); - GroupSymbol mappedSymbol = getMappedGroup(fakeSymbol); - - newGroupName = mappedSymbol.getName(); + + if (obj.getGroup() == null) { + return; } - // Finally, swap name of group, which should be the name of the group - // for all of the element symbols - obj.setShortName(newGroupName + ".*"); //$NON-NLS-1$ - + obj.setGroup(getMappedGroup(obj.getGroup())); } /** - * Swap each ElementSymbol referenced by AllSymbol - * @param obj Object to remap - */ - public void visit(AllSymbol obj) { - List oldSymbols = obj.getElementSymbols(); - if(oldSymbols != null && oldSymbols.size() > 0) { - List newSymbols = new ArrayList(oldSymbols.size()); - - Iterator iter = oldSymbols.iterator(); - while(iter.hasNext()) { - ElementSymbol es = (ElementSymbol) iter.next(); - newSymbols.add( getMappedElement(es) ); - } - - obj.setElementSymbols(newSymbols); - } - } - - /** * Swap group in unary from clause. * @param obj Object to remap */ Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -32,15 +32,21 @@ import org.teiid.query.sql.navigator.PreOrPostOrderNavigator; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.ElementSymbol; -import org.teiid.query.sql.symbol.SingleElementSymbol; +import org.teiid.query.sql.symbol.Expression; +import org.teiid.query.sql.symbol.WindowFunction; public class AggregateSymbolCollectorVisitor extends LanguageVisitor { public static class AggregateStopNavigator extends PreOrPostOrderNavigator { - - public AggregateStopNavigator(LanguageVisitor visitor) { - super(visitor, PreOrPostOrderNavigator.POST_ORDER, false); + + private Collection<? extends Expression> groupingCols; + private Collection<? super Expression> groupingColsUsed; + + public AggregateStopNavigator(LanguageVisitor visitor, Collection<? super Expression> groupingColsUsed, Collection<? extends Expression> groupingCols) { + super(visitor, PreOrPostOrderNavigator.PRE_ORDER, false); + this.groupingCols = groupingCols; + this.groupingColsUsed = groupingColsUsed; } public void visit(AggregateSymbol obj) { @@ -49,32 +55,56 @@ postVisitVisitor(obj); } + @Override + protected void visitNode(LanguageObject obj) { + if (groupingCols != null && obj instanceof Expression && groupingCols.contains(obj)) { + if (groupingColsUsed != null) { + groupingColsUsed.add((Expression)obj); + } + return; + } + super.visitNode(obj); + } + } private Collection<? super AggregateSymbol> aggregates; - private Collection<? super SingleElementSymbol> groupingSymbols; + private Collection<? super ElementSymbol> otherElements; + private Collection<? super WindowFunction> windowFunctions; - public AggregateSymbolCollectorVisitor(Collection<? super AggregateSymbol> aggregates, Collection<? super SingleElementSymbol> elements) { + public AggregateSymbolCollectorVisitor(Collection<? super AggregateSymbol> aggregates, Collection<? super ElementSymbol> elements) { this.aggregates = aggregates; - this.groupingSymbols = elements; + this.otherElements = elements; } public void visit(AggregateSymbol obj) { - if (aggregates != null) { + if (aggregates != null && !obj.isWindowed()) { this.aggregates.add(obj); } } + public void visit(WindowFunction windowFunction) { + if (this.windowFunctions != null) { + this.windowFunctions.add(windowFunction); + } + } + public void visit(ElementSymbol obj) { - if (this.groupingSymbols != null) { - this.groupingSymbols.add(obj); + if (this.otherElements != null && !obj.isExternalReference()) { + this.otherElements.add(obj); } } - public static final void getAggregates(LanguageObject obj, Collection<? super AggregateSymbol> aggregates, Collection<SingleElementSymbol> elements) { - AggregateSymbolCollectorVisitor visitor = new AggregateSymbolCollectorVisitor(aggregates, elements); - AggregateStopNavigator asn = new AggregateStopNavigator(visitor); - obj.acceptVisitor(asn); + public static final void getAggregates(LanguageObject obj, + Collection<? super AggregateSymbol> aggregates, + Collection<? super ElementSymbol> otherElements, + Collection<? super Expression> groupingColsUsed, + Collection<? super Expression> windowFunctions, + Collection<? extends Expression> groupingCols) { + AggregateSymbolCollectorVisitor visitor = new AggregateSymbolCollectorVisitor(aggregates, otherElements); + visitor.windowFunctions = windowFunctions; + AggregateStopNavigator asn = new AggregateStopNavigator(visitor, groupingColsUsed, groupingCols); + asn.visitNode(obj); } public static final Collection<AggregateSymbol> getAggregates(LanguageObject obj, boolean removeDuplicates) { @@ -88,7 +118,7 @@ aggregates = new ArrayList<AggregateSymbol>(); } AggregateSymbolCollectorVisitor visitor = new AggregateSymbolCollectorVisitor(aggregates, null); - AggregateStopNavigator asn = new AggregateStopNavigator(visitor); + AggregateStopNavigator asn = new AggregateStopNavigator(visitor, null, null); obj.acceptVisitor(asn); return aggregates; } Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -32,9 +32,8 @@ import org.teiid.query.sql.LanguageVisitor; import org.teiid.query.sql.navigator.DeepPreOrderNavigator; import org.teiid.query.sql.navigator.PreOrderNavigator; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.ElementSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; /** @@ -81,24 +80,13 @@ * called directly. * @param obj Language object */ - public void visit(AllInGroupSymbol obj) { + public void visit(MultipleElementSymbol obj) { if(obj.getElementSymbols() != null) { this.elements.addAll(obj.getElementSymbols()); } } /** - * Visit a language object and collect symbols. This method should NOT be - * called directly. - * @param obj Language object - */ - public void visit(AllSymbol obj) { - if(obj.getElementSymbols() != null) { - this.elements.addAll(obj.getElementSymbols()); - } - } - - /** * Helper to quickly get the elements from obj in the elements collection * @param obj Language object * @param elements Collection to collect elements in Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -109,8 +109,6 @@ import org.teiid.query.sql.proc.WhileStatement; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.CaseExpression; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.DerivedColumn; @@ -119,6 +117,7 @@ import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.QueryString; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; @@ -1157,14 +1156,16 @@ append(escapeSinglePart(obj.getOutputName())); } - public void visit( AllInGroupSymbol obj ) { - append(obj.getName()); + public void visit( MultipleElementSymbol obj ) { + if (obj.getGroup() == null) { + append(Tokens.ALL_COLS); + } else { + visitNode(obj.getGroup()); + append(Tokens.DOT); + append(Tokens.ALL_COLS); + } } - public void visit( AllSymbol obj ) { - append(obj.getName()); - } - public void visit( Constant obj ) { Class<?> type = obj.getType(); String[] constantParts = null; Deleted: trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -1,163 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.query.validator; - -import java.util.Collection; -import java.util.Set; - -import org.teiid.core.types.DataTypeManager; -import org.teiid.query.QueryPlugin; -import org.teiid.query.sql.LanguageObject; -import org.teiid.query.sql.navigator.PreOrderNavigator; -import org.teiid.query.sql.symbol.AggregateSymbol; -import org.teiid.query.sql.symbol.CaseExpression; -import org.teiid.query.sql.symbol.ElementSymbol; -import org.teiid.query.sql.symbol.Expression; -import org.teiid.query.sql.symbol.ExpressionSymbol; -import org.teiid.query.sql.symbol.Function; -import org.teiid.query.sql.symbol.SearchedCaseExpression; -import org.teiid.query.sql.symbol.AggregateSymbol.Type; -import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; -import org.teiid.query.sql.visitor.ElementCollectorVisitor; - -/** - * Validate that all ElementSymbol and ExpressionSymbols used in the HAVING - * and SELECT clauses are based on symbols used in the GROUP BY clause. - * @since 4.2 - */ -public class AggregateValidationVisitor extends AbstractValidationVisitor { - - private boolean validateBelow = true; - - // Symbols from GROUP BY - may be null if no group symbols - private Set<Expression> groupExpressions; - - public AggregateValidationVisitor(Set<Expression> groupExpressions) { - this.groupExpressions = groupExpressions; - } - - public void visit(AggregateSymbol obj) { - Expression aggExp = obj.getExpression(); - - validateNoNestedAggs(aggExp); - validateNoNestedAggs(obj.getOrderBy()); - validateNoNestedAggs(obj.getCondition()); - - // Verify data type of aggregate expression - Type aggregateFunction = obj.getAggregateFunction(); - if((aggregateFunction == Type.SUM || aggregateFunction == Type.AVG) && obj.getType() == null) { - handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0041", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ - } else if (obj.getType() != DataTypeManager.DefaultDataClasses.NULL) { - if (aggregateFunction == Type.XMLAGG && aggExp.getType() != DataTypeManager.DefaultDataClasses.XML) { - handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.non_xml", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ - } else if (obj.isBoolean() && aggExp.getType() != DataTypeManager.DefaultDataClasses.BOOLEAN) { - handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.non_boolean", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ - } - } - if((obj.isDistinct() || aggregateFunction == Type.MIN || aggregateFunction == Type.MAX) && DataTypeManager.isNonComparable(DataTypeManager.getDataTypeName(aggExp.getType()))) { - handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.non_comparable", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ - } - if(obj.isEnhancedNumeric()) { - if (!Number.class.isAssignableFrom(aggExp.getType())) { - handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0041", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ - } - if (obj.isDistinct()) { - handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.invalid_distinct", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ - } - } - validateBelow = false; - } - - private void validateNoNestedAggs(LanguageObject aggExp) { - // Check for any nested aggregates (which are not allowed) - if(aggExp != null) { - Collection<AggregateSymbol> nestedAggs = AggregateSymbolCollectorVisitor.getAggregates(aggExp, true); - if(nestedAggs.size() > 0) { - handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0039", nestedAggs), nestedAggs); //$NON-NLS-1$ - } - } - } - - public void visit(ElementSymbol obj) { - validateExpression(obj); - } - - public void visit(ExpressionSymbol obj) { - validateExpression(obj); - } - - public void visit(CaseExpression obj) { - validateExpression(obj); - } - - public void visit(SearchedCaseExpression obj) { - validateExpression(obj); - } - - public void visit(Function obj) { - validateExpression(obj); - } - - private void validateExpression(Expression symbol) { - if (ElementCollectorVisitor.getElements(symbol, false).isEmpty()) { - validateBelow = false; - return; - } - - if(groupExpressions == null) { - if (symbol instanceof ElementSymbol && !((ElementSymbol)symbol).isExternalReference()) { - handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0037", symbol), symbol); //$NON-NLS-1$ - } - } else if(! groupExpressions.contains(symbol)) { - if (symbol instanceof ElementSymbol && !((ElementSymbol)symbol).isExternalReference()) { - handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0038", symbol), symbol); //$NON-NLS-1$ - } - } else { - validateBelow = false; - } - } - - public static void validate(LanguageObject obj, final AggregateValidationVisitor visitor) { - visitor.validateBelow = true; - PreOrderNavigator nav = new PreOrderNavigator(visitor) { - private boolean validateBelow; - - protected void visitNode(LanguageObject obj) { - if (validateBelow) { - super.visitNode(obj); - } - } - - protected void preVisitVisitor(LanguageObject obj) { - super.preVisitVisitor(obj); - this.validateBelow = visitor.validateBelow; - } - - protected void postVisitVisitor(LanguageObject obj) { - this.validateBelow = true; - } - }; - obj.acceptVisitor(nav); - } - -} Modified: trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -27,6 +27,7 @@ import java.util.Collection; import java.util.HashSet; import java.util.Iterator; +import java.util.LinkedHashSet; import java.util.LinkedList; import java.util.List; import java.util.Map; @@ -121,6 +122,7 @@ import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.symbol.TextLine; +import org.teiid.query.sql.symbol.WindowFunction; import org.teiid.query.sql.symbol.XMLAttributes; import org.teiid.query.sql.symbol.XMLElement; import org.teiid.query.sql.symbol.XMLForest; @@ -752,42 +754,38 @@ validateNoAggsInClause(groupBy); validateNoAggsInClause(query.getCriteria()); validateNoAggsInClause(query.getFrom()); - if(groupBy != null || having != null || !AggregateSymbolCollectorVisitor.getAggregates(select, false).isEmpty()) { - Set<Expression> groupSymbols = null; - if(groupBy != null) { - groupSymbols = new HashSet<Expression>(); - for (final Iterator<Expression> iterator = groupBy.getSymbols().iterator(); iterator.hasNext();) { - final Expression element = iterator.next(); - groupSymbols.add(element); - } - } - - // Validate HAVING, if it exists - AggregateValidationVisitor visitor = new AggregateValidationVisitor(groupSymbols); - if(having != null) { - AggregateValidationVisitor.validate(having, visitor); - } - - // Validate SELECT - List<SingleElementSymbol> projectedSymbols = select.getProjectedSymbols(); - for (SingleElementSymbol symbol : projectedSymbols) { - AggregateValidationVisitor.validate(symbol, visitor); - } - - // Move items to this report - ValidatorReport report = visitor.getReport(); - Collection<ValidatorFailure> items = report.getItems(); - super.getReport().addItems(items); + Set<Expression> groupSymbols = null; + boolean hasAgg = false; + if (groupBy != null) { + groupSymbols = new HashSet<Expression>(groupBy.getSymbols()); + hasAgg = true; } + LinkedHashSet<Expression> invalid = new LinkedHashSet<Expression>(); + LinkedHashSet<Expression> invalidWindowFunctions = new LinkedHashSet<Expression>(); + LinkedList<AggregateSymbol> aggs = new LinkedList<AggregateSymbol>(); + if (having != null) { + AggregateSymbolCollectorVisitor.getAggregates(having, aggs, invalid, null, invalidWindowFunctions, groupSymbols); + hasAgg = true; + } + for (SingleElementSymbol symbol : select.getProjectedSymbols()) { + AggregateSymbolCollectorVisitor.getAggregates(symbol, aggs, invalid, null, null, groupSymbols); + } + if ((!aggs.isEmpty() || hasAgg) && !invalid.isEmpty()) { + handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0037", invalid), invalid); //$NON-NLS-1$ + } + if (!invalidWindowFunctions.isEmpty()) { + handleValidationError(QueryPlugin.Util.getString("SQLParser.window_only_top_level", invalidWindowFunctions), invalidWindowFunctions); //$NON-NLS-1$ + } } private void validateNoAggsInClause(LanguageObject clause) { if (clause == null) { return; } - Collection<AggregateSymbol> aggs = AggregateSymbolCollectorVisitor.getAggregates(clause, false); + LinkedHashSet<Expression> aggs = new LinkedHashSet<Expression>(); + AggregateSymbolCollectorVisitor.getAggregates(clause, aggs, null, null, aggs, null); if (!aggs.isEmpty()) { - handleValidationError(QueryPlugin.Util.getString("SQLParser.Aggregate_only_top_level", aggs), aggs); + handleValidationError(QueryPlugin.Util.getString("SQLParser.Aggregate_only_top_level", aggs), aggs); //$NON-NLS-1$ } } @@ -1204,18 +1202,56 @@ } @Override + public void visit(WindowFunction windowFunction) { + AggregateSymbol.Type type = windowFunction.getFunction().getAggregateFunction(); + switch (type) { + case RANK: + case DENSE_RANK: + case ROW_NUMBER: + if (windowFunction.getOrderBy() == null) { + handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.analytical_requires_order_by", windowFunction), windowFunction); //$NON-NLS-1$ + } + } + validateNoSubqueriesOrOuterReferences(windowFunction); + } + + @Override public void visit(AggregateSymbol obj) { if (obj.getCondition() != null) { Expression condition = obj.getCondition(); - if (!ValueIteratorProviderCollectorVisitor.getValueIteratorProviders(condition).isEmpty()) { - handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.filter_subquery", condition), condition); //$NON-NLS-1$ - } - for (ElementSymbol es : ElementCollectorVisitor.getElements(condition, false)) { - if (es.isExternalReference()) { - handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.filter_subquery", es), es); //$NON-NLS-1$ - } - } + validateNoSubqueriesOrOuterReferences(condition); } + Expression aggExp = obj.getExpression(); + + validateNoNestedAggs(aggExp); + validateNoNestedAggs(obj.getOrderBy()); + validateNoNestedAggs(obj.getCondition()); + + // Verify data type of aggregate expression + Type aggregateFunction = obj.getAggregateFunction(); + if((aggregateFunction == Type.SUM || aggregateFunction == Type.AVG) && obj.getType() == null) { + handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0041", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ + } else if (obj.getType() != DataTypeManager.DefaultDataClasses.NULL) { + if (aggregateFunction == Type.XMLAGG && aggExp.getType() != DataTypeManager.DefaultDataClasses.XML) { + handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.non_xml", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ + } else if (obj.isBoolean() && aggExp.getType() != DataTypeManager.DefaultDataClasses.BOOLEAN) { + handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.non_boolean", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ + } + } + if((obj.isDistinct() || aggregateFunction == Type.MIN || aggregateFunction == Type.MAX) && DataTypeManager.isNonComparable(DataTypeManager.getDataTypeName(aggExp.getType()))) { + handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.non_comparable", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ + } + if(obj.isEnhancedNumeric()) { + if (!Number.class.isAssignableFrom(aggExp.getType())) { + handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0041", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ + } + if (obj.isDistinct()) { + handleValidationError(QueryPlugin.Util.getString("AggregateValidationVisitor.invalid_distinct", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ + } + } + if (obj.isWindowed() && obj.getOrderBy() != null) { + handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0042", new Object[] {aggregateFunction, obj}), obj); //$NON-NLS-1$ + } if (obj.getAggregateFunction() != Type.TEXTAGG) { return; } @@ -1235,7 +1271,29 @@ } } } + + private void validateNoSubqueriesOrOuterReferences(Expression expr) { + if (!ValueIteratorProviderCollectorVisitor.getValueIteratorProviders(expr).isEmpty()) { + handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.filter_subquery", expr), expr); //$NON-NLS-1$ + } + for (ElementSymbol es : ElementCollectorVisitor.getElements(expr, false)) { + if (es.isExternalReference()) { + handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.filter_subquery", es), es); //$NON-NLS-1$ + } + } + } + private void validateNoNestedAggs(LanguageObject aggExp) { + // Check for any nested aggregates (which are not allowed) + if(aggExp != null) { + HashSet<Expression> nestedAggs = new LinkedHashSet<Expression>(); + AggregateSymbolCollectorVisitor.getAggregates(aggExp, nestedAggs, null, null, nestedAggs, null); + if(!nestedAggs.isEmpty()) { + handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0039", nestedAggs), nestedAggs); //$NON-NLS-1$ + } + } + } + private String[] validateQName(LanguageObject obj, String name) { try { return Name11Checker.getInstance().getQNameParts(name); Modified: trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj =================================================================== --- trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj 2011-07-21 19:28:44 UTC (rev 3323) @@ -1740,7 +1740,7 @@ query=query(info) | (<TABLE> name=id() { Query q = new Query(); - q.setSelect(new Select(Arrays.asList(new AllSymbol()))); + q.setSelect(new Select(Arrays.asList(new MultipleElementSymbol()))); q.setFrom(new From(Arrays.asList(new UnaryFromClause(new GroupSymbol(name))))); query = q; }) | @@ -1826,7 +1826,7 @@ [<ALL> | (<DISTINCT> {isDistinct=true;})] (<STAR> { - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); } | ( symbol=selectSymbol(info) { select.addSymbol(symbol); } @@ -1908,7 +1908,7 @@ } } -AllInGroupSymbol allInGroupSymbol() : +MultipleElementSymbol allInGroupSymbol() : { Token allInGroupToken = null; @@ -1916,7 +1916,7 @@ { allInGroupToken = <ALL_IN_GROUP> { - return new AllInGroupSymbol(normalizeId(allInGroupToken.image)); + return new MultipleElementSymbol(normalizeId(allInGroupToken.image.substring(0, allInGroupToken.image.length() - 2))); } } @@ -2773,7 +2773,7 @@ proc = storedProcedure(info, new StoredProcedure()) //deprecated ) { - subquery = new Query(new Select(Arrays.asList(new AllSymbol())), new From(Arrays.asList(new SubqueryFromClause("x", proc))), null, null, null); + subquery = new Query(new Select(Arrays.asList(new MultipleElementSymbol())), new From(Arrays.asList(new SubqueryFromClause("x", proc))), null, null, null); } ) <RPAREN> Modified: trunk/engine/src/main/resources/org/teiid/query/i18n.properties =================================================================== --- trunk/engine/src/main/resources/org/teiid/query/i18n.properties 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/main/resources/org/teiid/query/i18n.properties 2011-07-21 19:28:44 UTC (rev 3323) @@ -204,10 +204,10 @@ ERR.015.012.0033 = Metadata does not allow updates on the group: {0} ERR.015.012.0034 = Queries involving UNIONs, INTERSECTs and EXCEPTs not allowed on XML documents ERR.015.012.0035 = Queries combined with the set operator {0} must have the same number of output elements. -ERR.015.012.0037 = Invalid symbol: {0}. When an aggregate function is used in the SELECT clause and no GROUP BY clause is specified, the SELECT clause may contain only aggregate functions and constants. -ERR.015.012.0038 = Invalid symbol: {0}. When a GROUP BY clause is used, all elements in the SELECT clause must be declared in the GROUP BY clause. +ERR.015.012.0037 = {0} cannot be used outside of aggregate functions since they are not present in a GROUP BY clause. ERR.015.012.0039 = Nested aggregate expressions are not allowed: {0} ERR.015.012.0041 = The aggregate function {0} cannot be used with non-numeric expressions: {1} +ERR.015.012.0042 = The aggregate function {0} cannot use its own order by clause when windowed: {1} AggregateValidationVisitor.non_comparable = The aggregate function {0} cannot be used with non-comparable expressions: {1} AggregateValidationVisitor.non_xml = The XMLAGG aggregate function {0} requires an expression of type XML: {1} AggregateValidationVisitor.non_boolean=The boolean aggregate functions ANY, SOME, EVERY require a boolean expression. @@ -259,7 +259,8 @@ SQLParser.Unknown_join_type=Unknown join type: {0} -SQLParser.Aggregate_only_top_level=Aggregate functions are not allowed in the FROM/WHERE/GROUP BY clauses: {0} +SQLParser.Aggregate_only_top_level=Aggregate and window functions are not allowed in the FROM/WHERE/GROUP BY clauses: {0} +SQLParser.window_only_top_level=Window functions are not allowed in the HAVING clause: {0} SQLParser.Unknown_agg_func=Unknown aggregate function: {0} SQLParser.Invalid_func=Invalid function name: [{0}] SQLParser.Integer_parse=Unable to parse integer literal: {0} @@ -678,7 +679,8 @@ SimpleQueryResolver.procedure_cache_not_usable=Procedure caching will not be used for {0} since the result set cache is disabled or the results/parameters cannot be cached. SimpleQueryResolver.procedure_cache_not_used=Procedure caching will not be used for {0} due to the use of OPTION NOCACHE. ValidationVisitor.groupby_subquery=Expressions used in a GROUP BY clause cannot be constant and must not contain subqueries: "{0}". -ValidationVisitor.filter_subquery=Expressions used in a FILTER clause must not contain subqueries nor outer references: "{0}". +ValidationVisitor.filter_subquery=Expressions used in a FILTER clause or a window function must not contain subqueries nor outer references: "{0}". +ValidationVisitor.analytical_requires_order_by=The window analytical functions RANK, DENSE_RANK, and ROW_NUMBER require the use of an ORDER BY clause in the window specification: {0}. ValidationVisitor.Procedure_has_group_self_reference=Procedure cannot have a Group reference to itself. ExpressionEvaluator.Expected_props_for_payload_function=Unable to evaluate {0}: expected Properties for command payload but got object of type {1} ValidationVisitor.The_rowlimit_function_cannot_be_used_in_a_non-XML_command=The ''rowlimit'' and ''rowlimitexception'' functions cannot be used in a non-XML command Modified: trunk/engine/src/test/java/org/teiid/cdk/CommandBuilder.java =================================================================== --- trunk/engine/src/test/java/org/teiid/cdk/CommandBuilder.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/cdk/CommandBuilder.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -38,7 +38,7 @@ import org.teiid.query.sql.lang.Command; import org.teiid.query.sql.lang.Query; import org.teiid.query.sql.lang.Select; -import org.teiid.query.sql.symbol.AllSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; /** @@ -90,8 +90,8 @@ List expandedSymbols = new ArrayList(); for (Iterator i = originalSymbols.iterator(); i.hasNext(); ) { Object next = i.next(); - if (next instanceof AllSymbol) { - AllSymbol allSymbol = (AllSymbol) next; + if (next instanceof MultipleElementSymbol) { + MultipleElementSymbol allSymbol = (MultipleElementSymbol) next; expandedSymbols.addAll(allSymbol.getElementSymbols()); } else { expandedSymbols.add(next); Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -823,6 +823,11 @@ System.setProperty(RuleMergeCriteria.UNNEST_DEFAULT, Boolean.TRUE.toString()); TestQueryRewriter.helpTestRewriteCommand("Select e1 from pm3.g1 where pm3.g1.e2 < (select max(e2) FROM pm1.g1 WHERE pm3.g1.e1 = e1 HAVING min(e3) < pm3.g1.e3)", "SELECT e1 FROM pm3.g1, (SELECT MAX(e2) AS MAX, e1, MIN(e3) AS MIN FROM pm1.g1 GROUP BY e1) AS X__1 WHERE (X__1.MIN < pm3.g1.e3) AND (pm3.g1.e2 < X__1.MAX) AND (pm3.g1.e1 = X__1.e1)", RealMetadataFactory.example4()); } + + @Test public void testSubqueryRewriteToJoinWithGroupingExpression() throws Exception { + System.setProperty(RuleMergeCriteria.UNNEST_DEFAULT, Boolean.TRUE.toString()); + TestQueryRewriter.helpTestRewriteCommand("Select distinct e1 from pm3.g1 where exists (select 1 FROM pm1.g1 group by e4 || 'x' HAVING min(e3) || (e4 || 'x') = pm3.g1.e3)", "SELECT DISTINCT e1 FROM pm3.g1, (SELECT 1 AS EXPR, MIN(e3) AS MIN, concat(e4, 'x') AS EXPR_0, concat(MIN(e3), concat(e4, 'x')) AS EXPR_1 FROM pm1.g1 GROUP BY concat(e4, 'x')) AS X__1 WHERE pm3.g1.e3 = X__1.EXPR_1", RealMetadataFactory.example4()); + } /** * A join will not be used here because of the not Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/relational/rules/TestRulePlaceAccess.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/relational/rules/TestRulePlaceAccess.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/relational/rules/TestRulePlaceAccess.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -32,7 +32,7 @@ import org.teiid.query.sql.lang.From; import org.teiid.query.sql.lang.Query; import org.teiid.query.sql.lang.Select; -import org.teiid.query.sql.symbol.AllSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.unittest.RealMetadataFactory; @@ -67,7 +67,7 @@ query.setFrom(from); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); group.setMetadataID(METADATA.getGroupID("pm4.g2")); //$NON-NLS-1$ Modified: trunk/engine/src/test/java/org/teiid/query/parser/TestLimitParsing.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/parser/TestLimitParsing.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/parser/TestLimitParsing.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -34,7 +34,7 @@ import org.teiid.query.sql.lang.SetQuery; import org.teiid.query.sql.lang.UnaryFromClause; import org.teiid.query.sql.lang.SetQuery.Operation; -import org.teiid.query.sql.symbol.AllSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.Reference; @@ -43,7 +43,7 @@ @Test public void testLimit() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -53,7 +53,7 @@ @Test public void testLimitWithOffset() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -63,7 +63,7 @@ @Test public void testLimitWithReferences1() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -73,7 +73,7 @@ @Test public void testLimitWithReferences2() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -83,7 +83,7 @@ @Test public void testLimitWithReferences3() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -93,7 +93,7 @@ @Test public void testSetQueryLimit() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -104,7 +104,7 @@ @Test public void testOffset() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -114,7 +114,7 @@ @Test public void testFetchFirst() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -124,7 +124,7 @@ @Test public void testFetchFirstRow() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -134,7 +134,7 @@ @Test public void testOffsetFetch() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); Modified: trunk/engine/src/test/java/org/teiid/query/parser/TestOptionsAndHints.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/parser/TestOptionsAndHints.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/parser/TestOptionsAndHints.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -57,7 +57,7 @@ import org.teiid.query.sql.proc.HasCriteria; import org.teiid.query.sql.proc.IfStatement; import org.teiid.query.sql.proc.Statement; -import org.teiid.query.sql.symbol.AllSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; @@ -434,7 +434,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -452,7 +452,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -470,7 +470,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -488,7 +488,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -506,7 +506,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -524,7 +524,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -542,7 +542,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("t1")); //$NON-NLS-1$ @@ -560,7 +560,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -578,7 +578,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("t1")); //$NON-NLS-1$ @@ -596,13 +596,13 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); Query query2 = new Query(); select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query2.setSelect(select); From from2 = new From(); from2.addGroup(new GroupSymbol("t1")); //$NON-NLS-1$ @@ -623,7 +623,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); @@ -692,7 +692,7 @@ Query query2 = new Query(); select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from2 = new From(); from2.addGroup(new GroupSymbol("t1")); //$NON-NLS-1$ from2.addGroup(new GroupSymbol("t2")); //$NON-NLS-1$ @@ -1042,7 +1042,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -1058,7 +1058,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -1087,7 +1087,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -1106,7 +1106,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); UnaryFromClause ufc = new UnaryFromClause(); @@ -1116,7 +1116,7 @@ Query query1 = new Query(); select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query1.setSelect(select); from = new From(); ufc = new UnaryFromClause(); Modified: trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -101,8 +101,6 @@ import org.teiid.query.sql.proc.WhileStatement; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.CaseExpression; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.DerivedColumn; @@ -111,6 +109,7 @@ import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SearchedCaseExpression; @@ -217,7 +216,7 @@ From from = new From(); from.addClause(jp); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -237,7 +236,7 @@ From from = new From(); from.addClause(jp); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -259,7 +258,7 @@ from.addClause(jp); from.addClause(new UnaryFromClause(new GroupSymbol("g3"))); //$NON-NLS-1$ - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -286,7 +285,7 @@ From from = new From(); from.addClause(jp2); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -311,7 +310,7 @@ From from = new From(); from.addClause(jp3); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -334,7 +333,7 @@ From from = new From(); from.addClause(jp2); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -358,7 +357,7 @@ from.addClause(jp2); from.addClause(new UnaryFromClause(new GroupSymbol("g4"))); //$NON-NLS-1$ - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -387,7 +386,7 @@ from.addClause(g4); from.addClause(jp3); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -416,7 +415,7 @@ from.addClause(g1); from.addClause(jp); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); Select select = new Select(); select.addSymbol(all); @@ -1918,7 +1917,7 @@ from.addGroup(h); Select select = new Select(); - AllInGroupSymbol myG = new AllInGroupSymbol("myG.*"); //$NON-NLS-1$ + MultipleElementSymbol myG = new MultipleElementSymbol("myG"); //$NON-NLS-1$ select.addSymbol(myG); select.addSymbol(new ElementSymbol("myH.b")); //$NON-NLS-1$ @@ -1939,7 +1938,7 @@ from.addGroup(h); Select select = new Select(); - AllInGroupSymbol myG = new AllInGroupSymbol("myG.*"); //$NON-NLS-1$ + MultipleElementSymbol myG = new MultipleElementSymbol("myG"); //$NON-NLS-1$ select.addSymbol(myG); select.addSymbol(new ElementSymbol("myH.b")); //$NON-NLS-1$ @@ -2309,7 +2308,7 @@ from.addGroup(g); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); Query query = new Query(); query.setSelect(select); @@ -2327,7 +2326,7 @@ from.addGroup(g); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); Query query = new Query(); query.setSelect(select); @@ -2345,7 +2344,7 @@ from.addGroup(g); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); Query query = new Query(); query.setSelect(select); @@ -2676,10 +2675,10 @@ /** SELECT a FROM db.g WHERE b = aString order by c desc,d desc*/ @Test public void testOrderBysDesc(){ - ArrayList elements = new ArrayList(); + ArrayList<ElementSymbol> elements = new ArrayList<ElementSymbol>(); elements.add(new ElementSymbol("c")); //$NON-NLS-1$ elements.add(new ElementSymbol("d")); //$NON-NLS-1$ - ArrayList orderTypes = new ArrayList(); + ArrayList<Boolean> orderTypes = new ArrayList<Boolean>(); orderTypes.add(Boolean.FALSE); orderTypes.add(Boolean.FALSE); OrderBy orderBy = new OrderBy(elements, orderTypes); @@ -2692,10 +2691,10 @@ /** SELECT a FROM db.g WHERE b = aString order by c desc,d*/ @Test public void testMixedOrderBys(){ - ArrayList elements = new ArrayList(); + ArrayList<ElementSymbol> elements = new ArrayList<ElementSymbol>(); elements.add(new ElementSymbol("c")); //$NON-NLS-1$ elements.add(new ElementSymbol("d")); //$NON-NLS-1$ - ArrayList orderTypes = new ArrayList(); + ArrayList<Boolean> orderTypes = new ArrayList<Boolean>(); orderTypes.add(Boolean.FALSE); orderTypes.add(Boolean.TRUE); OrderBy orderBy = new OrderBy(elements, orderTypes); @@ -3358,7 +3357,7 @@ @Test public void testHasEQCriteria() throws Exception { ElementSymbol a = new ElementSymbol("a"); //$NON-NLS-1$ - List elements = new ArrayList(); + List<ElementSymbol> elements = new ArrayList<ElementSymbol>(); elements.add(a); CriteriaSelector critSelector = new CriteriaSelector(); @@ -3391,7 +3390,7 @@ /**HAS IN CRITERIA ON (a)*/ @Test public void testHasInCriteria() throws Exception { ElementSymbol a = new ElementSymbol("a"); //$NON-NLS-1$ - List elements = new ArrayList(); + List<ElementSymbol> elements = new ArrayList<ElementSymbol>(); elements.add(a); CriteriaSelector critSelector = new CriteriaSelector(); @@ -3408,7 +3407,7 @@ /**HAS COMPARE_LT CRITERIA ON (a)*/ @Test public void testHasLTCriteria() throws Exception { ElementSymbol a = new ElementSymbol("a"); //$NON-NLS-1$ - List elements = new ArrayList(); + List<ElementSymbol> elements = new ArrayList<ElementSymbol>(); elements.add(a); CriteriaSelector critSelector = new CriteriaSelector(); @@ -3425,7 +3424,7 @@ /**HAS COMPARE_LE CRITERIA ON (a)*/ @Test public void testHasLECriteria() throws Exception { ElementSymbol a = new ElementSymbol("a"); //$NON-NLS-1$ - List elements = new ArrayList(); + List<ElementSymbol> elements = new ArrayList<ElementSymbol>(); elements.add(a); CriteriaSelector critSelector = new CriteriaSelector(); @@ -3604,7 +3603,7 @@ * @return */ private Query sampleQuery() { - List symbols = new ArrayList(); + List<ElementSymbol> symbols = new ArrayList<ElementSymbol>(); symbols.add(new ElementSymbol("a1")); //$NON-NLS-1$ Select select = new Select(symbols); @@ -3645,7 +3644,7 @@ //sql is a variable, also uses the as, into, and update clauses @Test public void testDynamicCommandStatement1() throws Exception { - List symbols = new ArrayList(); + List<ElementSymbol> symbols = new ArrayList<ElementSymbol>(); ElementSymbol a1 = new ElementSymbol("a1"); //$NON-NLS-1$ a1.setType(DataTypeManager.DefaultDataClasses.STRING); @@ -3737,7 +3736,7 @@ Statement declStmt = new DeclareStatement(var1, shortType); //ifblock - List symbols = new ArrayList(); + List<ElementSymbol> symbols = new ArrayList<ElementSymbol>(); symbols.add(new ElementSymbol("a1")); //$NON-NLS-1$ Select select = new Select(symbols); @@ -3761,7 +3760,7 @@ ElementSymbol var2 = new ElementSymbol("var2"); //$NON-NLS-1$ Statement elseDeclStmt = new DeclareStatement(var2, shortType); - List elseSymbols = new ArrayList(); + List<ElementSymbol> elseSymbols = new ArrayList<ElementSymbol>(); elseSymbols.add(new ElementSymbol("b1")); //$NON-NLS-1$ Select elseSelect = new Select(elseSymbols); @@ -3770,7 +3769,6 @@ elseQuery.setFrom(from); elseQuery.setCriteria(criteria); - Command elseQueryCmd = elseQuery; AssignmentStatement elseQueryStmt = new AssignmentStatement(var2, elseQuery); Block elseBlock = new Block(); @@ -3867,7 +3865,7 @@ ElementSymbol var2 = new ElementSymbol("var2"); //$NON-NLS-1$ Statement elseDeclStmt = new DeclareStatement(var2, shortType); - List elseSymbols = new ArrayList(); + List<ElementSymbol> elseSymbols = new ArrayList<ElementSymbol>(); elseSymbols.add(new ElementSymbol("b1")); //$NON-NLS-1$ Select elseSelect = new Select(elseSymbols); @@ -5011,7 +5009,7 @@ StoredProcedure exec = new StoredProcedure(); exec.setProcedureName("m.sq1"); //$NON-NLS-1$ - Query query = new Query(new Select(Arrays.asList(new AllSymbol())), new From(Arrays.asList(new SubqueryFromClause("x", exec))), null, null, null); + Query query = new Query(new Select(Arrays.asList(new MultipleElementSymbol())), new From(Arrays.asList(new SubqueryFromClause("x", exec))), null, null, null); SubquerySetCriteria subCrit = new SubquerySetCriteria(expr, query); Query outer = new Query(); @@ -5079,7 +5077,7 @@ @Test public void testExecSubquery(){ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addClause(new UnaryFromClause(new GroupSymbol("newModel2.Table1"))); //$NON-NLS-1$ @@ -5102,7 +5100,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("TestDocument.TestDocument")); //$NON-NLS-1$ @@ -5122,7 +5120,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("TestDocument.TestDocument")); //$NON-NLS-1$ @@ -5165,7 +5163,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("a.thing")); //$NON-NLS-1$ @@ -5184,7 +5182,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("a.thing")); //$NON-NLS-1$ @@ -5203,7 +5201,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("a.thing")); //$NON-NLS-1$ @@ -5220,7 +5218,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); from.addGroup(new GroupSymbol("a.thing")); //$NON-NLS-1$ @@ -5265,7 +5263,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); @@ -5307,7 +5305,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); @@ -5353,7 +5351,7 @@ Query query = new Query(); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); query.setSelect(select); From from = new From(); @@ -5998,7 +5996,7 @@ @Test public void testAndOrPrecedence_1575() { Select s = new Select(); - s.addSymbol(new AllSymbol()); + s.addSymbol(new MultipleElementSymbol()); From f = new From(); f.addGroup(new GroupSymbol("m.g1")); //$NON-NLS-1$ CompareCriteria c1 = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(0))); //$NON-NLS-1$ @@ -6017,7 +6015,7 @@ @Test public void testAndOrPrecedence2_1575() { Select s = new Select(); - s.addSymbol(new AllSymbol()); + s.addSymbol(new MultipleElementSymbol()); From f = new From(); f.addGroup(new GroupSymbol("m.g1")); //$NON-NLS-1$ CompareCriteria c1 = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(0))); //$NON-NLS-1$ @@ -6040,7 +6038,7 @@ */ private void helpTestCompoundNonJoinCriteria(String sqlPred, PredicateCriteria predCrit) { Select s = new Select(); - s.addSymbol(new AllSymbol()); + s.addSymbol(new MultipleElementSymbol()); From f = new From(); CompareCriteria c1 = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(0))); //$NON-NLS-1$ @@ -6486,7 +6484,7 @@ Query query = new Query(); Select select = new Select(); query.setSelect(select); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); query.setFrom(from); Criteria compareCriteria = new CompareCriteria(new ElementSymbol("A.x"), CompareCriteria.EQ, new ElementSymbol("B.x")); //$NON-NLS-1$ //$NON-NLS-2$ @@ -6522,7 +6520,7 @@ String expected = "SELECT * FROM pm1.g1 UNION JOIN pm1.g2 WHERE g1.e1 = 1"; //$NON-NLS-1$ Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addClause(new JoinPredicate(new UnaryFromClause(new GroupSymbol("pm1.g1")), new UnaryFromClause(new GroupSymbol("pm1.g2")), JoinType.JOIN_UNION)); //$NON-NLS-1$ //$NON-NLS-2$ @@ -6696,7 +6694,7 @@ @Test public void testNestedTable() throws Exception { String sql = "SELECT * from TABLE(exec foo()) as x"; //$NON-NLS-1$ Query query = new Query(); - query.setSelect(new Select(Arrays.asList(new AllSymbol()))); + query.setSelect(new Select(Arrays.asList(new MultipleElementSymbol()))); StoredProcedure sp = new StoredProcedure(); sp.setProcedureName("foo"); SubqueryFromClause sfc = new SubqueryFromClause("x", sp); @@ -6708,7 +6706,7 @@ @Test public void testTextTable() throws Exception { String sql = "SELECT * from texttable(file columns x string WIDTH 1, y date width 10 skip 10) as x"; //$NON-NLS-1$ Query query = new Query(); - query.setSelect(new Select(Arrays.asList(new AllSymbol()))); + query.setSelect(new Select(Arrays.asList(new MultipleElementSymbol()))); TextTable tt = new TextTable(); tt.setFile(new ElementSymbol("file")); List<TextTable.TextColumn> columns = new ArrayList<TextTable.TextColumn>(); @@ -6738,7 +6736,7 @@ @Test public void testXMLTable() throws Exception { String sql = "SELECT * from xmltable(xmlnamespaces(no default), '/' columns x for ordinality, y date default {d'2000-01-01'} path '@date') as x"; //$NON-NLS-1$ Query query = new Query(); - query.setSelect(new Select(Arrays.asList(new AllSymbol()))); + query.setSelect(new Select(Arrays.asList(new MultipleElementSymbol()))); XMLTable xt = new XMLTable(); xt.setName("x"); xt.setNamespaces(new XMLNamespaces(Arrays.asList(new XMLNamespaces.NamespaceItem()))); @@ -6802,7 +6800,7 @@ Query query = new Query(); Select select = new Select(); query.setSelect(select); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("X")))); query.setFrom(from); helpTest("TABLE X", "SELECT * FROM X", query); @@ -6811,7 +6809,7 @@ @Test public void testArrayTable() throws Exception { String sql = "SELECT * from arraytable(null columns x string, y date) as x"; //$NON-NLS-1$ Query query = new Query(); - query.setSelect(new Select(Arrays.asList(new AllSymbol()))); + query.setSelect(new Select(Arrays.asList(new MultipleElementSymbol()))); ArrayTable tt = new ArrayTable(); tt.setArrayValue(new Constant(null, DataTypeManager.DefaultDataClasses.OBJECT)); List<TableFunctionReference.ProjectedColumn> columns = new ArrayList<TableFunctionReference.ProjectedColumn>(); Modified: trunk/engine/src/test/java/org/teiid/query/parser/TestSetQueryParsing.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/parser/TestSetQueryParsing.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/parser/TestSetQueryParsing.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -28,7 +28,7 @@ import org.teiid.query.sql.lang.Select; import org.teiid.query.sql.lang.SetQuery; import org.teiid.query.sql.lang.SetQuery.Operation; -import org.teiid.query.sql.symbol.AllSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.GroupSymbol; @@ -81,7 +81,7 @@ from.addGroup(g); Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); Query query1 = new Query(); query1.setSelect(select); Modified: trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/processor/xml/TestXMLProcessor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -7846,7 +7846,7 @@ "</Catalogs>\r\n\r\n"; //$NON-NLS-1$ - helpTestProcess("SELECT Catalogs.Catalog.Items.Item.ItemID, Order.* " + //$NON-NLS-1$ + helpTestProcess("SELECT Catalogs.Catalog.Items.Item.ItemID, \"Order\".* " + //$NON-NLS-1$ " FROM xmltest.doc9c " + //$NON-NLS-1$ " WHERE SupplierID > '54' " + //$NON-NLS-1$ " ORDER BY ItemID DESC, OrderID DESC", //$NON-NLS-1$ Modified: trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/resolver/TestResolver.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -93,7 +93,6 @@ import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.Reference; -import org.teiid.query.sql.symbol.SelectSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.visitor.CommandCollectorVisitor; import org.teiid.query.sql.visitor.ElementCollectorVisitor; @@ -293,11 +292,11 @@ private void helpCheckSelect(Query query, String[] elementNames) { Select select = query.getSelect(); - List elements = select.getSymbols(); + List<SingleElementSymbol> elements = select.getProjectedSymbols(); assertEquals("Wrong number of select symbols: ", elementNames.length, elements.size()); //$NON-NLS-1$ for(int i=0; i<elements.size(); i++) { - SelectSymbol symbol = (SelectSymbol) elements.get(i); + SingleElementSymbol symbol = elements.get(i); assertEquals("Element name does not match: ", elementNames[i].toUpperCase(), symbol.getName().toUpperCase()); //$NON-NLS-1$ } } @@ -594,7 +593,6 @@ @Test public void testSelectStar() { Query resolvedQuery = (Query) helpResolve("SELECT * FROM pm1.g1"); //$NON-NLS-1$ helpCheckFrom(resolvedQuery, new String[] { "pm1.g1" }); //$NON-NLS-1$ - helpCheckSelect(resolvedQuery, new String[] { "*" }); //$NON-NLS-1$ helpCheckElements(resolvedQuery.getSelect(), new String[] { "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4" }, //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ new String[] { "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4" } ); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ @@ -603,7 +601,6 @@ @Test public void testSelectStarFromAliasedGroup() { Query resolvedQuery = (Query) helpResolve("SELECT * FROM pm1.g1 as x"); //$NON-NLS-1$ helpCheckFrom(resolvedQuery, new String[] { "pm1.g1" }); //$NON-NLS-1$ - helpCheckSelect(resolvedQuery, new String[] { "*" }); //$NON-NLS-1$ helpCheckElements(resolvedQuery.getSelect(), new String[] { "x.e1", "x.e2", "x.e3", "x.e4" }, //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ new String[] { "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4" } ); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ @@ -612,7 +609,6 @@ @Test public void testSelectStarFromMultipleAliasedGroups() { Query resolvedQuery = (Query) helpResolve("SELECT * FROM pm1.g1 as x, pm1.g1 as y"); //$NON-NLS-1$ helpCheckFrom(resolvedQuery, new String[] { "pm1.g1", "pm1.g1" }); //$NON-NLS-1$ //$NON-NLS-2$ - helpCheckSelect(resolvedQuery, new String[] { "*" }); //$NON-NLS-1$ helpCheckElements(resolvedQuery.getSelect(), new String[] { "x.e1", "x.e2", "x.e3", "x.e4", "y.e1", "y.e2", "y.e3", "y.e4" }, //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$ //$NON-NLS-6$ //$NON-NLS-7$ //$NON-NLS-8$ new String[] { "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4", "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4" } ); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$ //$NON-NLS-6$ //$NON-NLS-7$ //$NON-NLS-8$ @@ -621,7 +617,6 @@ @Test public void testSelectStarWhereSomeElementsAreNotSelectable() { Query resolvedQuery = (Query) helpResolve("SELECT * FROM pm1.g4"); //$NON-NLS-1$ helpCheckFrom(resolvedQuery, new String[] { "pm1.g4" }); //$NON-NLS-1$ - helpCheckSelect(resolvedQuery, new String[] { "*" }); //$NON-NLS-1$ helpCheckElements(resolvedQuery.getSelect(), new String[] { "pm1.g4.e1", "pm1.g4.e3" }, //$NON-NLS-1$ //$NON-NLS-2$ new String[] { "pm1.g4.e1", "pm1.g4.e3" } ); //$NON-NLS-1$ //$NON-NLS-2$ @@ -630,7 +625,6 @@ @Test public void testSelectGroupStarWhereSomeElementsAreNotSelectable() { Query resolvedQuery = (Query) helpResolve("SELECT pm1.g4.* FROM pm1.g4"); //$NON-NLS-1$ helpCheckFrom(resolvedQuery, new String[] { "pm1.g4" }); //$NON-NLS-1$ - helpCheckSelect(resolvedQuery, new String[] { "pm1.g4.*" }); //$NON-NLS-1$ helpCheckElements(resolvedQuery.getSelect(), new String[] { "pm1.g4.e1", "pm1.g4.e3" }, //$NON-NLS-1$ //$NON-NLS-2$ new String[] { "pm1.g4.e1", "pm1.g4.e3" } ); //$NON-NLS-1$ //$NON-NLS-2$ @@ -639,7 +633,6 @@ @Test public void testFullyQualifiedSelectStar() { Query resolvedQuery = (Query) helpResolve("SELECT pm1.g1.* FROM pm1.g1"); //$NON-NLS-1$ helpCheckFrom(resolvedQuery, new String[] { "pm1.g1" }); //$NON-NLS-1$ - helpCheckSelect(resolvedQuery, new String[] { "pm1.g1.*" }); //$NON-NLS-1$ helpCheckElements(resolvedQuery.getSelect(), new String[] { "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4" }, //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ new String[] { "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4" } ); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ @@ -648,7 +641,6 @@ @Test public void testSelectAllInAliasedGroup() { Query resolvedQuery = (Query) helpResolve("SELECT x.* FROM pm1.g1 as x"); //$NON-NLS-1$ helpCheckFrom(resolvedQuery, new String[] { "pm1.g1" }); //$NON-NLS-1$ - helpCheckSelect(resolvedQuery, new String[] { "x.*" }); //$NON-NLS-1$ helpCheckElements(resolvedQuery.getSelect(), new String[] { "x.e1", "x.e2", "x.e3", "x.e4" }, //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ new String[] { "pm1.g1.e1", "pm1.g1.e2", "pm1.g1.e3", "pm1.g1.e4" } ); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ Modified: trunk/engine/src/test/java/org/teiid/query/sql/lang/TestSelect.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/sql/lang/TestSelect.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/sql/lang/TestSelect.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -28,7 +28,7 @@ import org.teiid.core.util.UnitTestUtil; import org.teiid.query.sql.lang.Select; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import junit.framework.TestCase; @@ -50,7 +50,7 @@ symbols.add(new ElementSymbol("b")); //$NON-NLS-1$ Select select = new Select(); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); all.setElementSymbols(symbols); select.addSymbol(all); return select; @@ -69,7 +69,7 @@ public void testGetProjectedNoElements() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); List projectedSymbols = select.getProjectedSymbols(); assertEquals("Did not get empty list for select * with no elements: ", new ArrayList(), projectedSymbols); //$NON-NLS-1$ @@ -81,7 +81,7 @@ symbols.add(new ElementSymbol("b")); //$NON-NLS-1$ Select select = new Select(); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); all.setElementSymbols(symbols); select.addSymbol(all); Modified: trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestAggregateSymbolCollectorVisitor.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestAggregateSymbolCollectorVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestAggregateSymbolCollectorVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -25,16 +25,15 @@ import java.util.ArrayList; import java.util.List; +import junit.framework.TestCase; + import org.teiid.api.exception.query.QueryParserException; import org.teiid.query.parser.QueryParser; import org.teiid.query.sql.lang.Command; -import org.teiid.query.sql.symbol.SingleElementSymbol; -import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; +import org.teiid.query.sql.symbol.Expression; -import junit.framework.TestCase; - public class TestAggregateSymbolCollectorVisitor extends TestCase { public TestAggregateSymbolCollectorVisitor(String name) { @@ -46,9 +45,9 @@ Command command = QueryParser.getQueryParser().parseCommand(sql); // Find aggregates - List<SingleElementSymbol> foundAggs = new ArrayList<SingleElementSymbol>(); - List<SingleElementSymbol> foundElements = new ArrayList<SingleElementSymbol>(); - AggregateSymbolCollectorVisitor.getAggregates(command, foundAggs, foundElements); + List<Expression> foundAggs = new ArrayList<Expression>(); + List<Expression> foundElements = new ArrayList<Expression>(); + AggregateSymbolCollectorVisitor.getAggregates(command, foundAggs, foundElements, null, null, null); // Compare assertEquals("Incorrect number of aggregates: " + foundAggs, aggs.length, foundAggs.size()); //$NON-NLS-1$ Modified: trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestSQLStringVisitor.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestSQLStringVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestSQLStringVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -77,8 +77,6 @@ import org.teiid.query.sql.proc.RaiseErrorStatement; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.CaseExpression; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; @@ -86,6 +84,7 @@ import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SearchedCaseExpression; @@ -571,7 +570,7 @@ public void testQuery1() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addGroup(new GroupSymbol("m.g")); //$NON-NLS-1$ Query query = new Query(); @@ -583,7 +582,7 @@ public void testQuery2() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addGroup(new GroupSymbol("m.g")); //$NON-NLS-1$ CompareCriteria cc = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(5))); //$NON-NLS-1$ @@ -606,7 +605,7 @@ public void testQuery3() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addGroup(new GroupSymbol("m.g")); //$NON-NLS-1$ GroupBy groupBy = new GroupBy(); @@ -627,7 +626,7 @@ public void testQuery4() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addGroup(new GroupSymbol("m.g")); //$NON-NLS-1$ CompareCriteria cc = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(5))); //$NON-NLS-1$ @@ -647,7 +646,7 @@ public void testQuery5() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addGroup(new GroupSymbol("m.g")); //$NON-NLS-1$ CompareCriteria cc = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(5))); //$NON-NLS-1$ @@ -668,7 +667,7 @@ public void testQuery6() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addGroup(new GroupSymbol("m.g")); //$NON-NLS-1$ CompareCriteria cc = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(5))); //$NON-NLS-1$ @@ -688,7 +687,7 @@ public void testQuery7() { Select select = new Select(); - select.addSymbol(new AllSymbol()); + select.addSymbol(new MultipleElementSymbol()); From from = new From(); from.addGroup(new GroupSymbol("m.g")); //$NON-NLS-1$ CompareCriteria cc = new CompareCriteria(new ElementSymbol("e1"), CompareCriteria.EQ, new Constant(new Integer(5))); //$NON-NLS-1$ @@ -1042,11 +1041,11 @@ } public void testAllSymbol() { - helpTest(new AllSymbol(), "*"); //$NON-NLS-1$ + helpTest(new MultipleElementSymbol(), "*"); //$NON-NLS-1$ } public void testAllInGroupSymbol() { - helpTest(new AllInGroupSymbol("m.g.*"), "m.g.*"); //$NON-NLS-1$ //$NON-NLS-2$ + helpTest(new MultipleElementSymbol("m.g"), "m.g.*"); //$NON-NLS-1$ //$NON-NLS-2$ } public void testConstantNull() { @@ -1851,7 +1850,7 @@ public void testLimit() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); @@ -1861,7 +1860,7 @@ public void testLimitWithOffset() { Query query = new Query(); - Select select = new Select(Arrays.asList(new AllSymbol())); + Select select = new Select(Arrays.asList(new MultipleElementSymbol())); From from = new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("a")))); //$NON-NLS-1$ query.setSelect(select); query.setFrom(from); Modified: trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestStaticSymbolMappingVisitor.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestStaticSymbolMappingVisitor.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestStaticSymbolMappingVisitor.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -28,6 +28,8 @@ import java.util.List; import java.util.Map; +import junit.framework.TestCase; + import org.teiid.query.sql.LanguageObject; import org.teiid.query.sql.lang.BetweenCriteria; import org.teiid.query.sql.lang.CompareCriteria; @@ -48,23 +50,17 @@ import org.teiid.query.sql.proc.CriteriaSelector; import org.teiid.query.sql.proc.TranslateCriteria; import org.teiid.query.sql.symbol.AliasSymbol; -import org.teiid.query.sql.symbol.AllInGroupSymbol; -import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; +import org.teiid.query.sql.symbol.MultipleElementSymbol; import org.teiid.query.sql.symbol.Symbol; -import org.teiid.query.sql.visitor.ElementCollectorVisitor; -import org.teiid.query.sql.visitor.GroupCollectorVisitor; -import org.teiid.query.sql.visitor.StaticSymbolMappingVisitor; -import junit.framework.TestCase; - public class TestStaticSymbolMappingVisitor extends TestCase { // ################################## FRAMEWORK ################################ @@ -209,14 +205,14 @@ public void testVisitSelect2() { Select select = new Select(); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); select.addSymbol(all); helpTest(select, getSymbolMap()); } public void testVisitSelect3() { Select select = new Select(); - AllSymbol all = new AllSymbol(); + MultipleElementSymbol all = new MultipleElementSymbol(); all.addElementSymbol(exampleElement(true, 0)); select.addSymbol(all); helpTest(select, getSymbolMap()); @@ -226,7 +222,7 @@ Select select = new Select(); select.addSymbol( new ExpressionSymbol( "x", new Function("length", new Expression[] {exampleElement(true, 0)})) ); //$NON-NLS-1$ //$NON-NLS-2$ - select.addSymbol( new AllInGroupSymbol("abc.*") ); //$NON-NLS-1$ + select.addSymbol( new MultipleElementSymbol("abc.*") ); //$NON-NLS-1$ select.addSymbol( exampleElement(true, 1) ); helpTest(select,getSymbolMap()); } @@ -266,7 +262,7 @@ } public void testVisitAllSymbol() { - AllSymbol as = new AllSymbol(); + MultipleElementSymbol as = new MultipleElementSymbol(); ArrayList elements = new ArrayList(); elements.add(exampleElement(true, 0)); elements.add(exampleElement(true, 1)); @@ -274,8 +270,8 @@ helpTest(as, getSymbolMap()); } - public void testVisitAllInGroupSymbol() { - AllInGroupSymbol aigs = new AllInGroupSymbol("OLDG0.*"); //$NON-NLS-1$ + public void testVisitMultipleElementSymbol() { + MultipleElementSymbol aigs = new MultipleElementSymbol("OLDG0.*"); //$NON-NLS-1$ ArrayList elements = new ArrayList(); elements.add(exampleElement(true, 0)); elements.add(exampleElement(true, 1)); @@ -299,9 +295,9 @@ helpTest(f2, getSymbolMap()); } - public void testMapAllInGroupSymbolName() { - AllInGroupSymbol aigs = new AllInGroupSymbol("OLDG0.*"); //$NON-NLS-1$ - ArrayList elements = new ArrayList(); + public void testMapMultipleElementSymbolName() { + MultipleElementSymbol aigs = new MultipleElementSymbol("OLDG0"); //$NON-NLS-1$ + ArrayList<ElementSymbol> elements = new ArrayList<ElementSymbol>(); elements.add(exampleElement(true, 0)); elements.add(exampleElement(true, 1)); aigs.setElementSymbols(elements); @@ -311,7 +307,7 @@ DeepPreOrderNavigator.doVisit(aigs, visitor); // Check name of all in group symbol - assertEquals("AllInGroupSymbol name did not get mapped correctly: ", "NEWG0.*", aigs.getName()); //$NON-NLS-1$ //$NON-NLS-2$ + assertEquals("MultipleElementSymbol name did not get mapped correctly: ", "NEWG0.*", aigs.toString()); //$NON-NLS-1$ //$NON-NLS-2$ } public void testExecName() { Modified: trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java 2011-07-21 16:29:29 UTC (rev 3322) +++ trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java 2011-07-21 19:28:44 UTC (rev 3323) @@ -1859,5 +1859,21 @@ @Test public void testNestedAgg() { helpValidate("SELECT min(g1.e1) filter (where max(e2) = 1) from pm1.g1", new String[] {"MAX(e2)"}, RealMetadataFactory.example1Cached()); } + + @Test public void testWindowFunction() { + helpValidate("SELECT e1 from pm1.g1 where row_number() over (order by e2) = 1", new String[] {"ROW_NUMBER() OVER (ORDER BY e2)"}, RealMetadataFactory.example1Cached()); + } + + @Test public void testWindowFunction1() { + helpValidate("SELECT 1 from pm1.g1 having row_number() over (order by e2) = 1", new String[] {"e2", "ROW_NUMBER() OVER (ORDER BY e2)"}, RealMetadataFactory.example1Cached()); + } + + @Test public void testWindowFunctionWithoutOrdering() { + helpValidate("SELECT row_number() over () from pm1.g1", new String[] {"ROW_NUMBER() OVER ()"}, RealMetadataFactory.example1Cached()); + } + @Test public void testWindowFunctionWithNestedOrdering() { + helpValidate("SELECT xmlagg(xmlelement(name x, e1) order by e2) over () from pm1.g1", new String[] {"XMLAGG(XMLELEMENT(NAME x, e1) ORDER BY e2)"}, RealMetadataFactory.example1Cached()); + } + } [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3322 - trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-21 12:29:29 -0400 (Thu, 21 Jul 2011) New Revision: 3322 Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleImplementJoinStrategy.java Log: TEIID-1678 fix for ordering position Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleImplementJoinStrategy.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/… [View More]RuleImplementJoinStrategy.java 2011-07-21 16:29:16 UTC (rev 3321) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleImplementJoinStrategy.java 2011-07-21 16:29:29 UTC (rev 3322) @@ -172,8 +172,6 @@ joinNode.setProperty(NodeConstants.Info.RIGHT_EXPRESSIONS, rightExpressions); } - Set<SingleElementSymbol> outputSymbols = new LinkedHashSet<SingleElementSymbol>((List<SingleElementSymbol>)joinNode.getProperty(NodeConstants.Info.OUTPUT_COLS)); - boolean pushedLeft = insertSort(joinNode.getFirstChild(), leftExpressions, joinNode, metadata, capabilitiesFinder, pushLeft); if (origExpressionCount == 1 @@ -236,6 +234,8 @@ } } + boolean sort = true; + if (sourceNode.getType() == NodeConstants.Types.ACCESS) { if (distinct || NewCalculateCostUtil.usesKey(sourceNode, expressions, metadata)) { joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.IS_LEFT_DISTINCT : NodeConstants.Info.IS_RIGHT_DISTINCT, true); @@ -248,13 +248,18 @@ } return true; } + } else if (sourceNode.getType() == NodeConstants.Types.GROUP) { + sourceNode.addAsParent(sortNode); + sort = false; // the grouping columns must contain all of the ordering columns } if (distinct) { joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.IS_LEFT_DISTINCT : NodeConstants.Info.IS_RIGHT_DISTINCT, true); } - joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.SORT_LEFT : NodeConstants.Info.SORT_RIGHT, SortOption.SORT); + if (sort) { + joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.SORT_LEFT : NodeConstants.Info.SORT_RIGHT, SortOption.SORT); + } if (needsCorrection) { PlanNode projectNode = NodeFactory.getNewNode(NodeConstants.Types.PROJECT); [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3321 - in branches/7.4.x/engine/src: test/java/org/teiid/query/processor and 1 other directory.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-21 12:29:16 -0400 (Thu, 21 Jul 2011) New Revision: 3321 Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleImplementJoinStrategy.java branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java Log: TEIID-1678 fix for ordering position Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleImplementJoinStrategy.java =====================================… [View More]============================== --- branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleImplementJoinStrategy.java 2011-07-21 14:16:11 UTC (rev 3320) +++ branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleImplementJoinStrategy.java 2011-07-21 16:29:16 UTC (rev 3321) @@ -172,8 +172,6 @@ joinNode.setProperty(NodeConstants.Info.RIGHT_EXPRESSIONS, rightExpressions); } - Set<SingleElementSymbol> outputSymbols = new LinkedHashSet<SingleElementSymbol>((List<SingleElementSymbol>)joinNode.getProperty(NodeConstants.Info.OUTPUT_COLS)); - boolean pushedLeft = insertSort(joinNode.getFirstChild(), leftExpressions, joinNode, metadata, capabilitiesFinder, pushLeft); if (origExpressionCount == 1 @@ -212,6 +210,10 @@ Set<SingleElementSymbol> orderSymbols = new LinkedHashSet<SingleElementSymbol>(expressions); PlanNode sourceNode = FrameUtil.findJoinSourceNode(childNode); + PlanNode parentGrouping = NodeEditor.findParent(sourceNode, NodeConstants.Types.GROUP, NodeConstants.Types.JOIN); + if (parentGrouping != null) { + sourceNode = parentGrouping; + } PlanNode joinNode = childNode.getParent(); Set<SingleElementSymbol> outputSymbols = new LinkedHashSet<SingleElementSymbol>((List<SingleElementSymbol>)childNode.getProperty(NodeConstants.Info.OUTPUT_COLS)); @@ -236,6 +238,8 @@ } } + boolean sort = true; + if (sourceNode.getType() == NodeConstants.Types.ACCESS) { if (distinct || NewCalculateCostUtil.usesKey(sourceNode, expressions, metadata)) { joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.IS_LEFT_DISTINCT : NodeConstants.Info.IS_RIGHT_DISTINCT, true); @@ -248,13 +252,18 @@ } return true; } + } else if (sourceNode.getType() == NodeConstants.Types.GROUP) { + sourceNode.addAsParent(sortNode); + sort = false; // the grouping columns must contain all of the ordering columns } if (distinct) { joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.IS_LEFT_DISTINCT : NodeConstants.Info.IS_RIGHT_DISTINCT, true); } - joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.SORT_LEFT : NodeConstants.Info.SORT_RIGHT, SortOption.SORT); + if (sort) { + joinNode.setProperty(joinNode.getFirstChild() == childNode ? NodeConstants.Info.SORT_LEFT : NodeConstants.Info.SORT_RIGHT, SortOption.SORT); + } if (needsCorrection) { PlanNode projectNode = NodeFactory.getNewNode(NodeConstants.Types.PROJECT); Modified: branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java =================================================================== --- branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java 2011-07-21 14:16:11 UTC (rev 3320) +++ branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java 2011-07-21 16:29:16 UTC (rev 3321) @@ -364,7 +364,7 @@ List<String> expectedQueries = new ArrayList<String>(6); for (int i = 0; i < 3; i++) { expectedQueries.add("SELECT g_0.id AS c_0, g_0.first AS c_1, g_0.last AS c_2 FROM CustomerMaster.Customers AS g_0 WHERE g_0.first = 'Miles' ORDER BY c_0"); //$NON-NLS-1$ - expectedQueries.add("SELECT g_0.id AS c_0, g_0.amount AS c_1 FROM Europe.CustAccts AS g_0 WHERE g_0.id = 100 ORDER BY c_0"); //$NON-NLS-1$ + expectedQueries.add("SELECT g_0.id, g_0.amount FROM Europe.CustAccts AS g_0 WHERE g_0.id = 100"); //$NON-NLS-1$ } assertEquals(expectedQueries, dataManager.getQueries()); [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3320 - in branches/7.4.x/engine/src: test/java/org/teiid/query/optimizer and 1 other directory.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-21 10:16:11 -0400 (Thu, 21 Jul 2011) New Revision: 3320 Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java Log: TEIID-1677 ensuring that exists does not unnest by default Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java =============================… [View More]====================================== --- branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java 2011-07-21 14:08:10 UTC (rev 3319) +++ branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java 2011-07-21 14:16:11 UTC (rev 3320) @@ -419,11 +419,11 @@ result.type = crit.getClass(); result.not = exists.isNegated(); //the correlations can only be in where (if no group by or aggregates) or having - result.query = (Query)exists.getCommand(); result.mergeJoin = exists.getSubqueryHint().isMergeJoin(); if (!UNNEST && !result.mergeJoin) { return result; } + result.query = (Query)exists.getCommand(); } return result; } Modified: branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java =================================================================== --- branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java 2011-07-21 14:08:10 UTC (rev 3319) +++ branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestSubqueryPushdown.java 2011-07-21 14:16:11 UTC (rev 3320) @@ -757,6 +757,7 @@ } @Test public void testSubqueryRewriteToJoinDistinct() throws Exception { + System.setProperty(RuleMergeCriteria.UNNEST_DEFAULT, Boolean.TRUE.toString()); TestQueryRewriter.helpTestRewriteCommand("Select distinct e1 from pm1.g1 as x where exists (select pm1.g1.e1 FROM pm1.g1 where e1 = x.e1)", "SELECT DISTINCT e1 FROM pm1.g1 AS x, (SELECT pm1.g1.e1 FROM pm1.g1) AS X__1 WHERE x.e1 = X__1.e1", RealMetadataFactory.example1Cached()); } @@ -764,6 +765,7 @@ * Agg does not depend on cardinality */ @Test public void testSubqueryRewriteToJoinGroupBy() throws Exception { + System.setProperty(RuleMergeCriteria.UNNEST_DEFAULT, Boolean.TRUE.toString()); TestQueryRewriter.helpTestRewriteCommand("Select max(e1) from pm1.g1 as x where exists (select pm1.g1.e1 FROM pm1.g1 where e1 = x.e1) group by e2", "SELECT MAX(e1) FROM pm1.g1 AS x, (SELECT pm1.g1.e1 FROM pm1.g1) AS X__1 WHERE x.e1 = X__1.e1 GROUP BY e2", RealMetadataFactory.example1Cached()); } @@ -775,6 +777,7 @@ } @Test public void testSubqueryRewriteToJoin() throws Exception { + System.setProperty(RuleMergeCriteria.UNNEST_DEFAULT, Boolean.TRUE.toString()); TestQueryRewriter.helpTestRewriteCommand("Select e1 from pm3.g1 where exists (select pm1.g1.e1 FROM pm1.g1 where e1 = pm3.g1.e1)", "SELECT e1 FROM pm3.g1, (SELECT pm1.g1.e1 FROM pm1.g1) AS X__1 WHERE pm3.g1.e1 = X__1.e1", RealMetadataFactory.example4()); } @@ -914,6 +917,7 @@ * Same as above, but the source is much larger, so a semi-join is favorable */ @Test public void testSemiJoinExistsCosting() { + System.setProperty(RuleMergeCriteria.UNNEST_DEFAULT, Boolean.TRUE.toString()); ProcessorPlan plan = helpPlan("Select e1 from pm2.g2 as o where not exists (select 1 from pm3.g1 where e1 = o.e1 having o.e2 = count(e2))", RealMetadataFactory.example4(), //$NON-NLS-1$ new String[] { "SELECT g_0.e1 AS c_0, g_0.e2 AS c_1 FROM pm2.g2 AS g_0 ORDER BY c_0, c_1" }); //$NON-NLS-1$ checkNodeTypes(plan, new int[] { [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3319 - in branches/7.4.x/engine/src: test/java/org/teiid/query/optimizer and 1 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-21 10:08:10 -0400 (Thu, 21 Jul 2011) New Revision: 3319 Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestProcessor.java Log: TEIID-1676 fix for … [View More]cardinality dependent aggregation Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java =================================================================== --- branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java 2011-07-21 03:40:15 UTC (rev 3318) +++ branches/7.4.x/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java 2011-07-21 14:08:10 UTC (rev 3319) @@ -645,7 +645,16 @@ Set<PlanNode> possibleTargetNodes = new LinkedHashSet<PlanNode>(aggregateMap.keySet()); possibleTargetNodes.addAll(groupingMap.keySet()); - + for (Map.Entry<PlanNode, List<AggregateSymbol>> entry : aggregateMap.entrySet()) { + if (AggregateSymbol.areAggregatesCardinalityDependent(entry.getValue())) { + //can't change the cardinality on the other side of the join - + //unless it's a 1-1 join, in which case this optimization isn't needed + //TODO: make a better choice if there are multiple targets + possibleTargetNodes.clear(); + possibleTargetNodes.add(entry.getKey()); + break; + } + } for (PlanNode planNode : possibleTargetNodes) { Set<SingleElementSymbol> stagedGroupingSymbols = new LinkedHashSet<SingleElementSymbol>(); List<AggregateSymbol> aggregates = aggregateMap.get(planNode); @@ -855,6 +864,10 @@ continue; } + if (originatingNode.getType() != NodeConstants.Types.ACCESS) { + continue; + } + if (aggs && ((AggregateSymbol)aggregateSymbol).isDistinct()) { //TODO: support distinct continue; Modified: branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java =================================================================== --- branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java 2011-07-21 03:40:15 UTC (rev 3318) +++ branches/7.4.x/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java 2011-07-21 14:08:10 UTC (rev 3319) @@ -69,7 +69,7 @@ String sql = "SELECT a12.intkey AS REGION_NBR, SUM(a11.intnum) AS WJXBFS1 FROM bqt1.smalla AS a11 INNER JOIN bqt2.smalla AS a12 ON a11.stringkey = a12.stringkey WHERE a11.stringkey = 0 GROUP BY a12.intkey"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.exampleBQTCached(), null, capFinder, - new String[] {"SELECT SUM(a11.intnum) FROM bqt1.smalla AS a11 WHERE a11.stringkey = '0' HAVING COUNT(*) > 0", "SELECT a12.intkey FROM bqt2.smalla AS a12 WHERE a12.stringkey = '0' group by a12.intkey"}, TestOptimizer.SHOULD_SUCCEED); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT SUM(a11.intnum) FROM bqt1.smalla AS a11 WHERE a11.stringkey = '0' HAVING COUNT(*) > 0", "SELECT a12.intkey FROM bqt2.smalla AS a12 WHERE a12.stringkey = '0'"}, TestOptimizer.SHOULD_SUCCEED); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -203,7 +203,7 @@ String sql = "select sum(a11.intnum) Profit, (sum(a11.intnum) / sum(a11.floatnum)) WJXBFS2 from bqt1.smalla a11 join bqt2.smallb a12 on a11.intkey=a12.intkey group by a12.intkey"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.exampleBQTCached(), null, capFinder, - new String[] {"SELECT g_0.intkey, SUM(g_0.intnum), SUM(g_0.floatnum) FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey", "SELECT g_0.intkey FROM bqt2.smallb AS g_0 GROUP BY g_0.intkey"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.intkey, SUM(g_0.intnum), SUM(g_0.floatnum) FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey", "SELECT g_0.intkey FROM bqt2.smallb AS g_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -298,7 +298,7 @@ String sql = "SELECT avg(y.e2) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, - new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0 GROUP BY g_0.e3, g_0.e2", "SELECT g_0.e3, g_0.e1, SUM(g_0.e2), COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0", "SELECT g_0.e3, g_0.e1, SUM(g_0.e2), COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -328,7 +328,7 @@ String sql = "SELECT stddev_pop(y.e2) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, - new String[] {"SELECT g_0.e3 AS c_0, g_0.e1 AS c_1, COUNT(g_0.e2) AS c_2, SUM(power(g_0.e2, 2)) AS c_3, SUM(g_0.e2) AS c_4 FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1 ORDER BY c_0", "SELECT g_0.e3 AS c_0, g_0.e2 AS c_1 FROM pm1.g1 AS g_0 GROUP BY g_0.e3, g_0.e2 ORDER BY c_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.e3 AS c_0, g_0.e1 AS c_1, COUNT(g_0.e2) AS c_2, SUM(power(g_0.e2, 2)) AS c_3, SUM(g_0.e2) AS c_4 FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1 ORDER BY c_0", "SELECT g_0.e3 AS c_0, g_0.e2 AS c_1 FROM pm1.g1 AS g_0 ORDER BY c_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -357,7 +357,7 @@ String sql = "SELECT count(y.e2) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, - new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0 GROUP BY g_0.e3, g_0.e2", "SELECT g_0.e3, g_0.e1, COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0", "SELECT g_0.e3, g_0.e1, COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -767,8 +767,8 @@ ProcessorPlan plan = helpPlan(sql, metadata, null, capFinder, - new String[] {"SELECT c2, c1, c0 FROM db2model.DB2TABLE", //$NON-NLS-1$ - "SELECT DISTINCT b2, sum(b0) FROM oraclemodel.OraTable GROUP BY b2 ORDER BY b2"}, //$NON-NLS-1$ + new String[] {"SELECT g_0.b2 AS c_0, g_0.b0 AS c_1 FROM oraclemodel.OraTable AS g_0 ORDER BY c_0", + "SELECT g_0.c2, g_0.c1, g_0.c0 FROM db2model.DB2TABLE AS g_0"}, //$NON-NLS-1$ SHOULD_SUCCEED ); checkNodeTypes(plan, new int[] { Modified: branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java =================================================================== --- branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java 2011-07-21 03:40:15 UTC (rev 3318) +++ branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java 2011-07-21 14:08:10 UTC (rev 3319) @@ -360,9 +360,9 @@ List[] expected = new List[] { Arrays.asList(null, "cc"), Arrays.asList(0.0, "bb"), - Arrays.asList(1.0, null), - Arrays.asList(2.0, "aa"), - Arrays.asList(7.0, "aa") + Arrays.asList(2.0, null), + Arrays.asList(21.0, "aa"), + Arrays.asList(24.0, "aa") }; // Construct data manager with data Modified: branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestProcessor.java =================================================================== --- branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestProcessor.java 2011-07-21 03:40:15 UTC (rev 3318) +++ branches/7.4.x/engine/src/test/java/org/teiid/query/processor/TestProcessor.java 2011-07-21 14:08:10 UTC (rev 3319) @@ -360,7 +360,7 @@ Properties props = new Properties(); props.setProperty("soap_host", "my.host.com"); //$NON-NLS-1$ //$NON-NLS-2$ props.setProperty("soap_port", "12345"); //$NON-NLS-1$ //$NON-NLS-2$ - CommandContext context = new CommandContext("0", "test", "user", null, "myvdb", 1, props, false); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ + CommandContext context = new CommandContext("0", "test", "user", null, "myvdb", 1, props, DEBUG); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ context.setProcessorBatchSize(BufferManager.DEFAULT_PROCESSOR_BATCH_SIZE); context.setConnectorBatchSize(BufferManager.DEFAULT_CONNECTOR_BATCH_SIZE); context.setBufferManager(BufferManagerFactory.getStandaloneBufferManager()); [View Less]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3318 - branches/7.4.x/engine/src/main/java/org/teiid/query/mapping/xml.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-20 23:40:15 -0400 (Wed, 20 Jul 2011) New Revision: 3318 Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/mapping/xml/MappingNode.java Log: ensuring that node properties are consistently ordered Modified: branches/7.4.x/engine/src/main/java/org/teiid/query/mapping/xml/MappingNode.java =================================================================== --- branches/7.4.x/engine/src/main/java/org/teiid/query/mapping/xml/MappingNode.java 2011-07-20 … [View More]

13 years, 9 months

1
0
0 / 0

teiid SVN: r3317 - in trunk: api/src/main/java/org/teiid/language/visitor and 33 other directories.

by teiid-commits＠lists.jboss.org

Author: shawkins Date: 2011-07-19 23:27:02 -0400 (Tue, 19 Jul 2011) New Revision: 3317 Added: trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java Removed: trunk/engine/src/main/java/org/teiid/core/id/IntegerID.java trunk/engine/src/main/java/org/teiid/core/id/IntegerIDFactory.java trunk/engine/src/main/java/org/teiid/core/id/InvalidIDException.java trunk/engine/src/main/java/org/teiid/core/id/LongID.java trunk/engine/src/main/java/org/teiid/core/id/… [View More]LongIDFactory.java trunk/engine/src/main/java/org/teiid/core/id/ObjectID.java trunk/engine/src/main/java/org/teiid/core/id/ObjectIDFactory.java trunk/engine/src/main/java/org/teiid/core/id/ParsedObjectID.java trunk/engine/src/main/java/org/teiid/core/id/StringID.java trunk/engine/src/main/java/org/teiid/core/id/StringIDFactory.java trunk/engine/src/main/java/org/teiid/core/id/UUIDFactory.java trunk/engine/src/test/java/org/teiid/core/id/ Modified: trunk/api/src/main/java/org/teiid/language/AggregateFunction.java trunk/api/src/main/java/org/teiid/language/visitor/HierarchyVisitor.java trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml trunk/engine/src/main/java/org/teiid/core/id/IDGenerator.java trunk/engine/src/main/java/org/teiid/core/id/UUID.java trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/plantree/NodeConstants.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/CapabilitiesUtil.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/FrameUtil.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/NewCalculateCostUtil.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleCollapseSource.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleConstants.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleDecomposeJoin.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeVirtual.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlaceAccess.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlanSorts.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushSelectCriteria.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseAccess.java trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseNull.java trunk/engine/src/main/java/org/teiid/query/parser/ParseInfo.java trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverUtil.java trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverVisitor.java trunk/engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/lang/GroupBy.java trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/AliasSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/ElementSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/symbol/ExpressionSymbol.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/ExpressionMappingVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/GroupsUsedByElementsVisitor.java trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj trunk/engine/src/main/resources/org/teiid/query/i18n.properties trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestExpressionsInGroupBy.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestInlineView.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestLimit.java trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java trunk/engine/src/test/java/org/teiid/query/optimizer/xml/TestXMLPlanner.java trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java trunk/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java trunk/engine/src/test/java/org/teiid/query/processor/TestProcessor.java trunk/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java trunk/engine/src/test/java/org/teiid/query/sql/symbol/TestSearchedCaseExpression.java trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestExpressionMappingVisitor.java trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java trunk/test-integration/common/src/test/java/org/teiid/dqp/internal/process/TestTPCR.java Log: TEIID-1676 TEIID-1667 changing base aggregate planning to use an symbolic planning form. beginnings of window function support. fix for cardinality dependent aggregate decomposition Modified: trunk/api/src/main/java/org/teiid/language/AggregateFunction.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/AggregateFunction.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/api/src/main/java/org/teiid/language/AggregateFunction.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -43,6 +43,7 @@ private String aggName; private boolean isDistinct; private Class<?> type; + private Expression condition; public AggregateFunction(String aggName, boolean isDistinct, Expression exp, Class<?> type) { this.expression = exp; @@ -118,5 +119,17 @@ public void setType(Class<?> type) { this.type = type; } + + /** + * + * @return the filter clause condition + */ + public Expression getCondition() { + return condition; + } + + public void setCondition(Expression condition) { + this.condition = condition; + } } Modified: trunk/api/src/main/java/org/teiid/language/visitor/HierarchyVisitor.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/visitor/HierarchyVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/api/src/main/java/org/teiid/language/visitor/HierarchyVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -79,6 +79,7 @@ public void visit(AggregateFunction obj) { visitNode(obj.getExpression()); + visitNode(obj.getCondition()); } public void visit(BatchedUpdates obj) { Modified: trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java =================================================================== --- trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/api/src/main/java/org/teiid/language/visitor/SQLStringVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -60,6 +60,7 @@ import org.teiid.language.Not; import org.teiid.language.OrderBy; import org.teiid.language.QueryExpression; +import org.teiid.language.SQLConstants; import org.teiid.language.ScalarSubquery; import org.teiid.language.SearchedCase; import org.teiid.language.SearchedWhenClause; @@ -174,11 +175,22 @@ } if (obj.getExpression() == null) { - buffer.append(Tokens.ALL_COLS); + if (SQLConstants.NonReserved.COUNT.equalsIgnoreCase(obj.getName())) { + buffer.append(Tokens.ALL_COLS); + } } else { append(obj.getExpression()); } buffer.append(Tokens.RPAREN); + if (obj.getCondition() != null) { + buffer.append(Tokens.SPACE); + buffer.append(FILTER); + buffer.append(Tokens.LPAREN); + buffer.append(WHERE); + buffer.append(Tokens.SPACE); + append(obj.getCondition()); + buffer.append(Tokens.RPAREN); + } } public void visit(Comparison obj) { Modified: trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java =================================================================== --- trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/api/src/main/java/org/teiid/translator/ExecutionFactory.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -860,5 +860,31 @@ public boolean supportsCommonTableExpressions() { return false; } + + /** + * @return true if Advanced OLAP operations are supporting + * including the aggregate function filter clause. + * @since 7.5 + */ + public boolean supportsAdvancedOlapOperations() { + return false; + } + + /** + * @return true if Elementary OLAP operations are supporting + * including window functions + * @since 7.5 + */ + public boolean supportsElementaryOlapOperations() { + return false; + } + + /** + * @return true if array_agg is supported + * @since 7.5 + */ + public boolean supportsArrayAgg() { + return false; + } } Modified: trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml =================================================================== --- trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/documentation/reference/src/main/docbook/en-US/content/sql_support.xml 2011-07-20 03:27:02 UTC (rev 3317) @@ -264,13 +264,22 @@ </para> </listitem> <listitem> - <para>Aggregate functions may only be used in the HAVING or SELECT clauses and may not be nested within another aggregate function. + <para>Aggregate functions cannot be used in FROM, GROUP BY, or WHERE clauses without an intervening query expression. </para> </listitem> <listitem> + <para>Aggregate functions cannot be nested within another aggregate function without an intervening query expression. + </para> + </listitem> + <listitem> <para>Aggregate functions may be nested inside other functions. </para> </listitem> + <listitem> + <para>Any aggregate function may take an optional FILTER clasue of the form <synopsis label="Usage">FILTER ( WHERE condition )</synopsis>. The condition may be any boolean value expression that does not contain a subquery or a correlated variable. + The filter will logically be evaluated for each row prior to the grouping operation. If false the aggregate function will not accumulate a value for the given row. + </para> + </listitem> </itemizedlist> <para> For more information on aggregates, see the sections on GROUP BY or HAVING. Modified: trunk/engine/src/main/java/org/teiid/core/id/IDGenerator.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/IDGenerator.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/IDGenerator.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -22,306 +22,17 @@ package org.teiid.core.id; -import java.util.Collection; -import java.util.HashMap; -import java.util.HashSet; -import java.util.Map; -import java.util.Set; -import org.teiid.core.CorePlugin; - - /** * IDGenerator */ public class IDGenerator { - private static final IDGenerator INSTANCE = new IDGenerator(); - - /** - * Obtain the shared instance of this class. - * @return the statically shared instance of this class. - */ - public static IDGenerator getInstance() { - return INSTANCE; - } - - /** ObjectIDFactory instances keyed by protocol */ - private final Map factories; - private final Set protocols; + private int id; + + public int nextInt() { + return id++; + } - private ObjectIDFactory defaultFactory; - - public IDGenerator() { - this.factories = new HashMap(); - this.protocols = new HashSet(); - - // Initialize default factory ... - final ObjectIDFactory newDefaultFactory = new UUIDFactory(); - addFactory( newDefaultFactory ); - if ( !this.hasDefaultFactory() ) { - this.setDefaultFactory(newDefaultFactory); - } - } - - /** - * Method that creates and adds to this generator all the built-in factories, and if there is no default - * factory, set the default factory to the {@link UUIDFactory}. - * This method may be called multiple times without side effect. - */ - public void addBuiltInFactories() { - // Add the UUID factory as the default ... - final ObjectIDFactory newDefaultFactory = new UUIDFactory(); - addFactory( newDefaultFactory ); - if ( !this.hasDefaultFactory() ) { - this.setDefaultFactory(newDefaultFactory); - } - addFactory( new IntegerIDFactory() ); - addFactory( new LongIDFactory() ); - addFactory( new StringIDFactory() ); - } - - /** - * Supply to this generator a new factory for a type of {@link ObjectID}. - * This method has no effect if the factory is null, or if this generator already knows - * about the factory. - * @param factory the new factory - */ - public void addFactory( final ObjectIDFactory factory ) { - if ( factory == null ) { - return; - } - final String protocol = factory.getProtocol(); - - // See if the factory is already known ... - if ( !this.factories.containsKey(protocol) ) { - this.factories.put(protocol,factory); - } - this.protocols.add(protocol); - } - - /** - * Remove a factory from this generator. - * This method has no effect if the supplied protocol doesn't match the protocol of any descriptor - * known to this generator. - * @param protocol the protocol for which the factory is to be removed - * @return whether a factory was found and removed for the supplied protocol - */ - public boolean removeFactory(String protocol) { - if ( protocol == null ) { - return false; - } - final Object previous = this.factories.remove(protocol); - if ( previous != null ) { - this.protocols.remove(protocol); - return true; - } - return false; - } - - /** - * Method to obtain the collection of {@link ObjectIDFactory} instances that each describe - * one of the types of {@link ObjectID}s that are available to this generator. - * @return the collection of {@link ObjectIDFactory} instances. - */ - public Collection getFactories() { - // Currently, we get the collection of values from the 'factories' map. This is okay, - // because although not terribly efficient, this method is not intended to be called very - // frequently - return factories.values(); - } - - /** - * Method to obtain the collection of {@link ObjectIDFactory} instances that each describe - * one of the types of {@link ObjectID}s that are available to this generator. - * @return the collection of {@link ObjectIDFactory} instances. - */ - public ObjectIDFactory getFactory( final String protocol ) { - return (ObjectIDFactory) this.factories.get(protocol); - } - - /** - * Method to obtain the set of {@link java.lang.String String} protocols. This is a utility that merely obtains the - * protocols from the factories. - * @return the Set of String protocols; never null - */ - public Set getProtocols() { - return factories.keySet(); - } - - /** - * Return whether there is a factory that is used by default for the {@link #create()} method is invoked. - * @return true if there is a default factory, or false otherwise - */ - public boolean hasDefaultFactory() { - return defaultFactory != null; - } - - /** - * Get the factory that is used by default for the {@link #create()} method is invoked. - * @return the default factory, or null if there is no default factory - */ - public ObjectIDFactory getDefaultFactory() { - return defaultFactory; - } - - /** - * Set the factory that should be used by default for the {@link #create()} method is invoked. - * @param factory the factory that should be used by default; may be null if no default is to be allowed. - */ - public void setDefaultFactory(final ObjectIDFactory factory) { - defaultFactory = factory; - } - - /** - * Set the factory that should be used by default for the {@link #create()} method is invoked. - * @param protocol the protocol for factory that should be used by default; may be null if no default - * is to be allowed. - */ - public void setDefaultFactory(final String protocol) { - final ObjectIDFactory factory = getFactory(protocol); // null if protocol doesn't match - defaultFactory = factory; - } - - /** - * Create a new {@link ObjectID} using the default factory - * @return the new ObjectID - */ - public ObjectID create() { - if ( this.defaultFactory != null ) { - return this.defaultFactory.create(); - } - // Invalid protocol - throw new IllegalArgumentException(CorePlugin.Util.getString("IDGenerator.No_default_id_factory_has_been_defined")); //$NON-NLS-1$ - } - - /** - * Create a new {@link ObjectID} for the type specified by the protocol - * @param protocol the protocol of the type of {@link ObjectID} to be created; may not be null - * @return the new ObjectID - */ - public ObjectID create( final String protocol ) { - if (protocol == null) { - final String msg = CorePlugin.Util.getString("IDGenerator.The_protocol_may_not_be_null"); //$NON-NLS-1$ - throw new IllegalArgumentException(msg); - } - // Get the factory ... - final ObjectIDFactory factory = (ObjectIDFactory) this.factories.get(protocol); - if ( factory != null ) { - // Create the new ObjectID ... - return factory.create(); - } - // Invalid protocol - throw new IllegalArgumentException(CorePlugin.Util.getString("IDGenerator.The_specified_ObjectID_protocol___8",protocol)); //$NON-NLS-1$ - } - - /** - * Convenience method for obtaining the stringified form of an ObjectID. - * @param id - * @return - */ - public String toString( final ObjectID id ) { - return id.toString(); - } - - /** - * Convenience method for obtaining the stringified form of an ObjectID. - * @param id - * @return - */ - public String toString( final ObjectID id, final char delim ) { - return id.toString(delim); - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param id the stringified id of the form <code>protocol:value</code>, where - * <code>protocol</code> defines the protocol of the ID, and <code>value</code - * contains the global identifier value; may never be null - * @return the ObjectID instance for the stringified ID - * @throws InvalidIDException if the specified string does not contain a valid ObjectID - * or if the protocol is unknown - */ - public ObjectID stringToObject(String id) throws InvalidIDException { - if (id == null) { - final String msg = CorePlugin.Util.getString("IDGenerator.The_stringified_ID_may_not_be_null"); //$NON-NLS-1$ - throw new IllegalArgumentException(msg); - } - - // Parse the string - final ParsedObjectID parsedID = ParsedObjectID.parsedStringifiedObjectID(id,this.protocols); - - // Find the appropriate factory and parse the id ... - final ObjectIDFactory factory = (ObjectIDFactory) this.factories.get(parsedID.getProtocol()); - ObjectID result = null; - if ( factory != null ) { - result = factory.stringWithoutProtocolToObject(parsedID.getRemainder()); - } - if ( result == null ) { - throw new InvalidIDException(CorePlugin.Util.getString("IDGenerator.The_stringified_ObjectID_has_an_unknown_protocol___16") + parsedID.getProtocol()); //$NON-NLS-1$ - } - return result; - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param id the stringified id of the form <code>protocol:value</code>, where - * <code>protocol</code> defines the protocol of the ID, and <code>value</code - * contains the global identifier value; may never be null - * @return the ObjectID instance for the stringified ID - * @throws InvalidIDException if the specified string does not contain a valid ObjectID - * or if the protocol is unknown - */ - public ObjectID stringToObject(String id, String protocol) throws InvalidIDException { - if (id == null) { - final String msg = CorePlugin.Util.getString("IDGenerator.The_stringified_ID_may_not_be_null"); //$NON-NLS-1$ - throw new IllegalArgumentException(msg); - } - - // Parse the string - final ParsedObjectID parsedID = ParsedObjectID.parsedStringifiedObjectID(id,protocol); - - // Find the appropriate factory and parse the id ... - final ObjectIDFactory factory = (ObjectIDFactory) this.factories.get(parsedID.getProtocol()); - ObjectID result = null; - if ( factory != null ) { - result = factory.stringWithoutProtocolToObject(parsedID.getRemainder()); - } - if ( result == null ) { - throw new InvalidIDException(CorePlugin.Util.getString("IDGenerator.The_stringified_ObjectID_has_an_unknown_protocol___16") + parsedID.getProtocol()); //$NON-NLS-1$ - } - return result; - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param id the stringified id of the form <code>protocol:value</code>, where - * <code>protocol</code> defines the protocol of the ID, and <code>value</code - * contains the global identifier value; may never be null - * @return the ObjectID instance for the stringified ID - * @throws InvalidIDException if the specified string does not contain a valid ObjectID - * or if the protocol is unknown - */ - public ObjectID stringToObject(String id, char delim) throws InvalidIDException { - if (id == null) { - final String msg = CorePlugin.Util.getString("IDGenerator.The_stringified_ID_may_not_be_null"); //$NON-NLS-1$ - throw new IllegalArgumentException(msg); - } - - // Parse the string - final ParsedObjectID parsedID = ParsedObjectID.parsedStringifiedObjectID(id,delim); - - // Find the appropriate factory and parse the id ... - final ObjectIDFactory factory = (ObjectIDFactory) this.factories.get(parsedID.getProtocol()); - ObjectID result = null; - if ( factory != null ) { - result = factory.stringWithoutProtocolToObject(parsedID.getRemainder()); - } - if ( result == null ) { - throw new InvalidIDException(CorePlugin.Util.getString("IDGenerator.The_stringified_ObjectID_has_an_unknown_protocol___16") + parsedID.getProtocol()); //$NON-NLS-1$ - } - return result; - } - } Deleted: trunk/engine/src/main/java/org/teiid/core/id/IntegerID.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/IntegerID.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/IntegerID.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,125 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import java.io.Serializable; - -import org.teiid.core.util.Assertion; - - -public class IntegerID implements ObjectID, Serializable { - - public static final String PROTOCOL = "mmint"; //$NON-NLS-1$ - private int id; - - protected IntegerID(int id) { - this.id = id; - } - /** - * Get hash code for object - * @return Hash code - */ - public int hashCode() { - return this.id; - } - - /** - /** - * Returns true if the specified object is semantically equal to this instance. - * Note: this method is consistent with <code>compareTo()</code>. - * - * @param obj the object that this instance is to be compared to. - * @return whether the object is equal to this object. - */ - public boolean equals(Object obj) { - // Check if instances are identical ... - if ( this == obj ) { - return true; - } - - // Check if object can be compared to this one - // (this includes checking for null ) ... - //if ( this.getClass().isInstance(obj) ) { - if ( obj instanceof IntegerID ) { - IntegerID that = (IntegerID) obj; - return ( this.id == that.id ); - } - - // Otherwise not comparable ... - return false; - } - /** - * Compares this object to another. If the specified object is not an instance of - * the LongID class, then this method throws a - * ClassCastException (as instances are comparable only to instances of the same - * class). - * Note: this method is consistent with <code>equals()</code>, meaning - * that <code>(compare(x, y)==0) == (x.equals(y))</code>. - * - * @param obj the object that this instance is to be compared to; may not be null. - * @return a negative integer, zero, or a positive integer as this object - * is less than, equal to, or greater than the specified object, respectively. - * @throws ClassCastException if the specified object's type prevents it - * from being compared to this instance. - */ - public int compareTo(Object obj) { - IntegerID that = (IntegerID) obj; // May throw ClassCastException - Assertion.isNotNull(obj); - - long diff = this.id - that.id; - if ( diff < 0 ) { - return -1; - } - if ( diff > 0 ) { - return 1; - } - return 0; - } - - /** - * Returns a string representing the current state of the object. - * @return the string representation of this instance. - */ - public String toString(){ - return PROTOCOL + ObjectID.DELIMITER + this.id; - } - /** - * @see org.teiid.core.id.ObjectID#toString(char) - */ - public String toString(char delim) { - return PROTOCOL + delim + this.id; - } - - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - public String getProtocol() { - return PROTOCOL; - } - - public int getValue() { - return this.id; - } -} - Deleted: trunk/engine/src/main/java/org/teiid/core/id/IntegerIDFactory.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/IntegerIDFactory.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/IntegerIDFactory.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,116 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import java.io.Serializable; - -import org.teiid.core.CorePlugin; - -public class IntegerIDFactory implements ObjectIDFactory, Serializable { - private int lastID = 0; - public IntegerIDFactory() { - } - - /** - * Return the description for the type of ObjectID described by this object. - * @return the description - */ - public String getDescription() { - return CorePlugin.Util.getString("IntegerIDFactory.Description"); //$NON-NLS-1$ - } - - protected int getNextValue() { - return ++lastID; - } - - /** - * Create a new ObjectID instance using this protocol. - * @return the new instance - */ - public ObjectID create(){ - return new IntegerID( getNextValue() ); - } - /** - * Return whether the specified ObjectID instance is valid. Only ObjectID instances - * that are for this protocol will be passed in. - * - * This implementation only checks whether the ObjectID is an instance of a LongID. - * @param id the ID that is to be validated, and which is never null - * @return true if the instance is valid for this protocol, or false if - * it is not valid. - */ - public boolean validate(ObjectID id) { - if ( id instanceof IntegerID ) { - return true; - } - return false; - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param value the stringified id (the result of {@link ObjectID#toString()}), - * and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringToObject(String value) throws InvalidIDException { - final ParsedObjectID parsedID = ParsedObjectID.parsedStringifiedObjectID(value,IntegerID.PROTOCOL); - try { - return new IntegerID( Integer.parseInt(parsedID.getRemainder()) ); - } catch ( NumberFormatException e ) { - throw new InvalidIDException(CorePlugin.Util.getString("IntegerIDFactory.The_specified_ID_value_is_invalid",value,getProtocol())); //$NON-NLS-1$ - } - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * This method is called by the {@link IDGenerator#stringToObject(String)} method, which - * must process the protocol to determine the correct parser to use. As such, it guarantees - * that the parser that receives this call can assume that the protocol was equal to the - * protocol returned by the parser's {@link ObjectIDDescriptor#getProtocol()}. - * @param value the stringified id with the protocol and ObjectID.DELIMITER already - * removed, and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringWithoutProtocolToObject(String value) throws InvalidIDException { - try { - return new IntegerID( Integer.parseInt(value) ); - } catch ( NumberFormatException e ) { - throw new InvalidIDException(CorePlugin.Util.getString("IntegerIDFactory.The_specified_ID_value_is_invalid",value,getProtocol())); //$NON-NLS-1$ - } - } - - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - public String getProtocol() { - return IntegerID.PROTOCOL; - } -} - Deleted: trunk/engine/src/main/java/org/teiid/core/id/InvalidIDException.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/InvalidIDException.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/InvalidIDException.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,70 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import org.teiid.core.CorePlugin; -import org.teiid.core.TeiidException; - -/** - * Exception which occurs if an error occurs within the server that is not - * business-related. For instance, if a service or bean is not available - * or communication fails. - */ -public class InvalidIDException extends TeiidException { - private static final String INVALID_ID_MESSAGE = CorePlugin.Util.getString("InvalidIDException.Invalid_ID_1"); //$NON-NLS-1$ - - /** - * No-Arg Constructor - */ - public InvalidIDException( ) { - super( ); - } - - - /** - * Construct an instance of InvalidIDException. - * @param message - */ - public InvalidIDException(String message) { - super(message); - } - - /** - * Construct an instance of InvalidIDException. - * @param e - */ - public InvalidIDException(Throwable e) { - super(e,INVALID_ID_MESSAGE); - } - - - /** - * Construct an instance of InvalidIDException. - * @param e - * @param message - */ - public InvalidIDException(Throwable e, String message) { - super(e, message); - } - -} Deleted: trunk/engine/src/main/java/org/teiid/core/id/LongID.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/LongID.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/LongID.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,126 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import java.io.Serializable; - -import org.teiid.core.util.Assertion; - - -public class LongID implements ObjectID, Serializable { - public static final String PROTOCOL = "mmlong"; //$NON-NLS-1$ - - private long id; - - protected LongID(long id) { - this.id = id; - } - /** - * Get hash code for object - * @return Hash code - */ - public int hashCode() { - return (int) this.id; - } - - /** - /** - * Returns true if the specified object is semantically equal to this instance. - * Note: this method is consistent with <code>compareTo()</code>. - * - * @param obj the object that this instance is to be compared to. - * @return whether the object is equal to this object. - */ - public boolean equals(Object obj) { - // Check if instances are identical ... - if ( this == obj ) { - return true; - } - - // Check if object can be compared to this one - // (this includes checking for null ) ... - //if ( this.getClass().isInstance(obj) ) { - if ( obj instanceof LongID ) { - LongID that = (LongID) obj; - return ( this.id == that.id ); - } - - // Otherwise not comparable ... - return false; - } - /** - * Compares this object to another. If the specified object is not an instance of - * the LongID class, then this method throws a - * ClassCastException (as instances are comparable only to instances of the same - * class). - * Note: this method is consistent with <code>equals()</code>, meaning - * that <code>(compare(x, y)==0) == (x.equals(y))</code>. - * - * @param obj the object that this instance is to be compared to; may not be null. - * @return a negative integer, zero, or a positive integer as this object - * is less than, equal to, or greater than the specified object, respectively. - * @throws ClassCastException if the specified object's type prevents it - * from being compared to this instance. - */ - public int compareTo(Object obj) { - LongID that = (LongID) obj; // May throw ClassCastException - Assertion.isNotNull(obj); - - long diff = this.id - that.id; - if ( diff < 0 ) { - return -1; - } - if ( diff > 0 ) { - return 1; - } - return 0; - } - - /** - * Returns a string representing the current state of the object. - * @return the string representation of this instance. - */ - public String toString(){ - return PROTOCOL + ObjectID.DELIMITER + this.id; - } - - /** - * @see org.teiid.core.id.ObjectID#toString(char) - */ - public String toString(char delim) { - return PROTOCOL + delim + this.id; - } - - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - public String getProtocol() { - return PROTOCOL; - } - - public long getValue() { - return this.id; - } -} - Deleted: trunk/engine/src/main/java/org/teiid/core/id/LongIDFactory.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/LongIDFactory.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/LongIDFactory.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,117 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import java.io.Serializable; - -import org.teiid.core.CorePlugin; - - -public class LongIDFactory implements ObjectIDFactory, Serializable { - private long lastID = 0; - public LongIDFactory() { - } - - /** - * Return the description for the type of ObjectID described by this object. - * @return the description - */ - public String getDescription() { - return CorePlugin.Util.getString("LongIDFactory.Description"); //$NON-NLS-1$ - } - - protected long getNextValue() { - return ++lastID; - } - - /** - * Create a new ObjectID instance using this protocol. - * @return the new instance - */ - public ObjectID create(){ - return new LongID( getNextValue() ); - } - /** - * Return whether the specified ObjectID instance is valid. Only ObjectID instances - * that are for this protocol will be passed in. - * - * This implementation only checks whether the ObjectID is an instance of a LongID. - * @param id the ID that is to be validated, and which is never null - * @return true if the instance is valid for this protocol, or false if - * it is not valid. - */ - public boolean validate(ObjectID id) { - if ( id instanceof LongID ) { - return true; - } - return false; - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * This method is called by the {@link IDGenerator#stringToObject(String)} method, which - * must process the protocol to determine the correct parser to use. As such, it guarantees - * that the parser that receives this call can assume that the protocol was equal to the - * protocol returned by the parser's {@link ObjectIDDescriptor#getProtocol()}. - * @param value the stringified id with the protocol and ObjectID.DELIMITER already - * removed, and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringWithoutProtocolToObject(String value) throws InvalidIDException { - try { - return new LongID( Long.parseLong(value) ); - } catch ( NumberFormatException e ) { - throw new InvalidIDException(CorePlugin.Util.getString("LongIDFactory.The_specified_ID_value_is_invalid",value,getProtocol())); //$NON-NLS-1$ - } - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param value the stringified id (the result of {@link ObjectID#toString()}), - * and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringToObject(String value) throws InvalidIDException { - final ParsedObjectID parsedID = ParsedObjectID.parsedStringifiedObjectID(value,LongID.PROTOCOL); - try { - return new LongID( Long.parseLong(parsedID.getRemainder()) ); - } catch ( NumberFormatException e ) { - throw new InvalidIDException(CorePlugin.Util.getString("LongIDFactory.The_specified_ID_value_is_invalid",value,getProtocol())); //$NON-NLS-1$ - } - } - - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - public String getProtocol() { - return LongID.PROTOCOL; - } -} - Deleted: trunk/engine/src/main/java/org/teiid/core/id/ObjectID.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/ObjectID.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/ObjectID.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,63 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - - - -public interface ObjectID extends Comparable { - - char DELIMITER = ':'; - - /** - * Compares this object to another. If the specified object is not an instance of - * the ObjectID class, then this method throws a - * ClassCastException (as instances are comparable only to instances of the same - * class). - * Note: this method is consistent with <code>equals()</code>, meaning - * that <code>(compare(x, y)==0) == (x.equals(y))</code>. - * - * @param obj the object that this instance is to be compared to; may not be null. - * @return a negative integer, zero, or a positive integer as this object - * is less than, equal to, or greater than the specified object, respectively. - * @throws ClassCastException if the specified object's type prevents it - * from being compared to this instance. - */ - int compareTo(Object obj); - /** - * Returns a string representing the current state of the object. - * @return the string representation of this instance. - */ - String toString(); - /** - * Returns a string representing the current state of the object. - * @param delim the delimiter that should be used - * @return the string representation of this instance. - */ - String toString( char delim ); - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - String getProtocol(); -} - Deleted: trunk/engine/src/main/java/org/teiid/core/id/ObjectIDFactory.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/ObjectIDFactory.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/ObjectIDFactory.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,76 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - - -/** - * The ObjectIDDescriptor defines the interface that is used to describe a particular type of - * {@link ObjectID}. - */ -public interface ObjectIDFactory { - - /** - * Return the protocol for the type of ObjectID described by this object. - * @return the protocol - */ - String getProtocol(); - - /** - * Return the description for the type of ObjectID described by this object. - * @return the description - */ - String getDescription(); - - /** - * Create a new GlobalUID instance using this protocol. - * @return the new instance - */ - ObjectID create(); - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param value the stringified id (the result of {@link ObjectID#toString()}), - * and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - ObjectID stringToObject(String value) throws InvalidIDException; - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * This method is called by the {@link IDGenerator#stringToObject(String)} method, which - * must process the protocol to determine the correct parser to use. As such, it guarantees - * that the parser that receives this call can assume that the protocol was equal to the - * protocol returned by the parser's {@link ObjectIDDescriptor#getProtocol()}. - * @param value the stringified id with the protocol and ObjectID.DELIMITER already - * removed, and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - ObjectID stringWithoutProtocolToObject(String value) throws InvalidIDException; - -} Deleted: trunk/engine/src/main/java/org/teiid/core/id/ParsedObjectID.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/ParsedObjectID.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/ParsedObjectID.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,88 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; -import java.util.Iterator; -import java.util.Set; - -import org.teiid.core.CorePlugin; - - -/** - * ParsedObjectID - */ -public class ParsedObjectID { - private static final int DELIMITER_LENGTH = 1; - private final String protocol; - private final String remainder; - - protected ParsedObjectID( final String protocol, final String remainder ) { - this.protocol = protocol; - this.remainder = remainder; - } - - public String getProtocol() { - return protocol; - } - - public String getRemainder() { - return remainder; - } - - public static ParsedObjectID parsedStringifiedObjectID( final String id, final Set protocols ) throws InvalidIDException { - final Iterator iter = protocols.iterator(); - while (iter.hasNext()) { - final String protocol = (String)iter.next(); - if ( id.startsWith(protocol) ) { - return new ParsedObjectID(protocol,id.substring(protocol.length()+DELIMITER_LENGTH)); - } - } - throw new InvalidIDException(CorePlugin.Util.getString("ParsedObjectID.The_stringified_ObjectID_does_not_have_a_protocol")); //$NON-NLS-1$ - } - - public static ParsedObjectID parsedStringifiedObjectID( final String id ) throws InvalidIDException { - return parsedStringifiedObjectID(id,ObjectID.DELIMITER); - } - - public static ParsedObjectID parsedStringifiedObjectID( final String id, final char delim ) throws InvalidIDException { - int index = -1; - if ( id != null ) { - // Obtain the protocol from the string - index = id.indexOf(delim); - } - if ( index == -1 ) { - throw new InvalidIDException(CorePlugin.Util.getString("ParsedObjectID.The_stringified_ObjectID_does_not_have_a_protocol")); //$NON-NLS-1$ - } - return new ParsedObjectID(id.substring(0,index),id.substring(index+DELIMITER_LENGTH)); - } - - public static ParsedObjectID parsedStringifiedObjectID( final String id, final String expectedProtocol ) throws InvalidIDException { - if ( expectedProtocol == null ) { - return new ParsedObjectID("",id); //$NON-NLS-1$ - } - if ( id.startsWith(expectedProtocol) ) { - return new ParsedObjectID(expectedProtocol,id.substring(expectedProtocol.length()+DELIMITER_LENGTH)); - } - throw new InvalidIDException(CorePlugin.Util.getString("ParsedObjectID.The_stringified_ObjectID_does_not_have_the_required_protocol_{0}",expectedProtocol)); //$NON-NLS-1$ - } - -} Deleted: trunk/engine/src/main/java/org/teiid/core/id/StringID.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/StringID.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/StringID.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,120 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import java.io.Serializable; - -import org.teiid.core.util.Assertion; - - -public class StringID implements ObjectID, Serializable { - public static final String PROTOCOL = "strid"; //$NON-NLS-1$ - - private String id; - - protected StringID(String id) { - this.id = id; - } - protected StringID(long id) { - this.id = Long.toString(id); - } - /** - * Get hash code for object - * @return Hash code - */ - public int hashCode() { - return id.hashCode(); - } - - /** - /** - * Returns true if the specified object is semantically equal to this instance. - * Note: this method is consistent with <code>compareTo()</code>. - * - * @param obj the object that this instance is to be compared to. - * @return whether the object is equal to this object. - */ - public boolean equals(Object obj) { - // Check if instances are identical ... - if ( this == obj ) { - return true; - } - - // Check if object can be compared to this one - // (this includes checking for null ) ... - //if ( this.getClass().isInstance(obj) ) { - if ( obj instanceof StringID ) { - StringID that = (StringID) obj; - return ( this.id.equals( that.id ) ); - } - - // Otherwise not comparable ... - return false; - } - /** - * Compares this object to another. If the specified object is not an instance of - * the StringID class, then this method throws a - * ClassCastException (as instances are comparable only to instances of the same - * class). - * Note: this method is consistent with <code>equals()</code>, meaning - * that <code>(compare(x, y)==0) == (x.equals(y))</code>. - * - * @param obj the object that this instance is to be compared to; may not be null. - * @return a negative integer, zero, or a positive integer as this object - * is less than, equal to, or greater than the specified object, respectively. - * @throws ClassCastException if the specified object's type prevents it - * from being compared to this instance. - */ - public int compareTo(Object obj) { - StringID that = (StringID) obj; // May throw ClassCastException - Assertion.isNotNull(obj); - return this.id.compareTo(that.id); - } - - /** - * Returns a string representing the current state of the object. - * @return the string representation of this instance. - */ - public String toString(){ - return PROTOCOL + ObjectID.DELIMITER + this.id; - } - /** - * @see org.teiid.core.id.ObjectID#toString(char) - */ - public String toString(char delim) { - return PROTOCOL + delim + this.id; - } - - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - public String getProtocol() { - return PROTOCOL; - } - - protected String getValue() { - return this.id; - } -} - Deleted: trunk/engine/src/main/java/org/teiid/core/id/StringIDFactory.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/StringIDFactory.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/StringIDFactory.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,116 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import org.teiid.core.CorePlugin; - - - -public class StringIDFactory implements ObjectIDFactory { - private long lastID = 0; - public StringIDFactory() { - } - - /** - * Return the description for the type of ObjectID described by this object. - * @return the description - */ - public String getDescription() { - return CorePlugin.Util.getString("StringIDFactory.Description"); //$NON-NLS-1$ - } - - protected long getNextValue() { - return ++lastID; - } - - /** - * Create a new ObjectID instance using this protocol. - * @return the new instance - */ - public ObjectID create(){ - return new StringID( getNextValue() ); - } - /** - * Return whether the specified ObjectID instance is valid. Only ObjectID instances - * that are for this protocol will be passed in. - * - * This implementation only checks whether the ObjectID is an instance of a StringID. - * @param id the ID that is to be validated, and which is never null - * @return true if the instance is valid for this protocol, or false if - * it is not valid. - */ - public boolean validate(ObjectID id) { - if ( id instanceof StringID ) { - return true; - } - return false; - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * This method is called by the {@link IDGenerator#stringToObject(String)} method, which - * must process the protocol to determine the correct parser to use. As such, it guarantees - * that the parser that receives this call can assume that the protocol was equal to the - * protocol returned by the parser's {@link ObjectIDDescriptor#getProtocol()}. - * @param value the stringified id with the protocol and ObjectID.DELIMITER already - * removed, and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringWithoutProtocolToObject(String value) throws InvalidIDException { - try { - return new StringID( value ); - } catch ( NumberFormatException e ) { - throw new InvalidIDException(CorePlugin.Util.getString("StringIDFactory.The_specified_ID_value_is_invalid",value,getProtocol())); //$NON-NLS-1$ - } - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param value the stringified id (the result of {@link ObjectID#toString()}), - * and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringToObject(String value) throws InvalidIDException { - final ParsedObjectID parsedID = ParsedObjectID.parsedStringifiedObjectID(value,LongID.PROTOCOL); - try { - return new StringID( parsedID.getRemainder() ); - } catch ( NumberFormatException e ) { - throw new InvalidIDException(CorePlugin.Util.getString("StringIDFactory.The_specified_ID_value_is_invalid",value,getProtocol())); //$NON-NLS-1$ - } - } - - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - public String getProtocol() { - return StringID.PROTOCOL; - } -} - Modified: trunk/engine/src/main/java/org/teiid/core/id/UUID.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/UUID.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/UUID.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -24,10 +24,7 @@ import java.io.Serializable; -import org.teiid.core.CorePlugin; -import org.teiid.core.util.Assertion; - /** * This class represents a universally unique identifier, consisting of * two long integral values. @@ -35,10 +32,11 @@ * This identifier is supposed to be unique both spatially and temporally. * It is based on version 4 IETF variant random UUIDs. */ -public class UUID implements ObjectID, Serializable { +public class UUID implements Serializable { + + private static final long serialVersionUID = 4730187208307761197L; - private static final String NOT_UUID_MESSAGE = CorePlugin.Util.getString("UUID.ID_must_be_of_type_UUID_1"); //$NON-NLS-1$ - private static final String UNPARSABLE_MESSAGE = CorePlugin.Util.getString("UUID.ID_must_be_of_type_UUID_to_parse_2"); //$NON-NLS-1$ + char DELIMITER = ':'; /** * The variants allowed by the UUID specification. */ @@ -78,42 +76,6 @@ } /** - * Return the first part of the UUID as a long. - * @return first part of the UUID as a long - */ - public static long getPart1(ObjectID id) { - Assertion.assertTrue((id instanceof UUID), UNPARSABLE_MESSAGE); - - UUID uuid = (UUID)id; - return uuid.uuid.getMostSignificantBits(); - } - - /** - * Return the first part of the UUID as a long. - * @return first part of the UUID as a long - */ - public static long getPart2(ObjectID id) { - Assertion.assertTrue((id instanceof UUID), UNPARSABLE_MESSAGE); - - UUID uuid = (UUID)id; - return uuid.uuid.getLeastSignificantBits(); - } - - public static int getVariant(ObjectID id ) { - Assertion.assertTrue((id instanceof UUID), NOT_UUID_MESSAGE); - - UUID uuid = (UUID)id; - return uuid.uuid.variant(); - } - - public static int getVersion(ObjectID id ) { - Assertion.assertTrue((id instanceof UUID), NOT_UUID_MESSAGE); - - UUID uuid = (UUID)id; - return uuid.uuid.version(); - } - - /** * Return the name of the protocol that this factory uses. * @return the protocol name */ @@ -179,7 +141,7 @@ * @return the string representation of this instance. */ public String toString(){ - return toString(ObjectID.DELIMITER); + return toString(DELIMITER); } /** @@ -206,20 +168,4 @@ return this.cachedExportableFormUuidString; } - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param value the stringified id with the protocol and ObjectID.DELIMITER already - * removed, and which is never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - */ - public static ObjectID stringToObject(String value) throws InvalidIDException { - try { - return new UUID(java.util.UUID.fromString(value)); - } catch ( IllegalArgumentException e ) { - throw new InvalidIDException( - CorePlugin.Util.getString("UUID.InvalidFormatForProtocol",value,PROTOCOL)); //$NON-NLS-1$ - } - } - } Deleted: trunk/engine/src/main/java/org/teiid/core/id/UUIDFactory.java =================================================================== --- trunk/engine/src/main/java/org/teiid/core/id/UUIDFactory.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/core/id/UUIDFactory.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1,132 +0,0 @@ -/* - * JBoss, Home of Professional Open Source. - * See the COPYRIGHT.txt file distributed with this work for information - * regarding copyright ownership. Some portions may be licensed - * to Red Hat, Inc. under one or more contributor license agreements. - * - * This library is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA - * 02110-1301 USA. - */ - -package org.teiid.core.id; - -import org.teiid.core.CorePlugin; - -/** - * This class is a factory for generating universally unique identifiers - * (UUID's). - * - * <h3>Output format for UUIDs</h3> - * UUIDs are output in the following 36-character format: - * <pre> - * xxxxxxxx-yyyy-zzzz-cccc-nnnnnnnnnnnn - * </pre> - * where x=least significant time component, y=middle significant time component, - * z=most significant time component multiplexed with version, c=clock sequence - * multiplexed with variant, and n=node component (random number). - * - * - * The generated ID's conform somewhat to the (now expired) IETF internet - * draft standard, "UUIDs and GUIDs", DCE spec on UUIDs. - * - * <ul> - * <li> - * <a href="http://hegel.ittc.ukans.edu/topics/internet/internet-drafts/draft-l/draft..."> - * UUIDs and GUIDs, P. Leach, R. Salz, 02/05/1998</a> - * </li> - * <li> - * <a href="http://www.opengroup.org/onlinepubs/009629399/apdxa.htm"> - * DCE Universal Unique Identifier</a>. - * </li> - * </ul> - * - * All references in this code to bit positions as "least significant" and - * "most significant" refer to the bits moving from right to left, respectively. - * - */ -public class UUIDFactory implements ObjectIDFactory { - - // ------------------------------------------------------------------------- - // C O N S T R U C T O R - // ------------------------------------------------------------------------- - - public UUIDFactory() { - } - - // ------------------------------------------------------------------------- - // P U B L I C M E T H O D S - // ------------------------------------------------------------------------- - - /** - * Return the description for the type of ObjectID described by this object. - * @return the description - */ - public String getDescription() { - return CorePlugin.Util.getString("UUIDFactory.Description"); //$NON-NLS-1$ - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * This method is called by the {@link IDGenerator#stringToObject(String)} method, which - * must process the protocol to determine the correct parser to use. As such, it guarantees - * that the parser that receives this call can assume that the protocol was equal to the - * protocol returned by the parser's {@link ObjectIDDescriptor#getProtocol()}. - * @param value the stringified id with the protocol and ObjectID.DELIMITER already - * removed, and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringWithoutProtocolToObject(String value) throws InvalidIDException { - return UUID.stringToObject(value); - } - - /** - * Attempt to convert the specified string to the appropriate ObjectID instance. - * @param value the stringified id (the result of {@link ObjectID#toString()}), - * and should never null or zero length - * @return the ObjectID instance for the stringified ID if this factory is able - * to parse the string, or null if the factory is unaware of the specified format. - * @throws InvalidIDException if the parser is aware of this protocol, but it is of the wrong - * format for this type of ObjectID. - */ - public ObjectID stringToObject(String value) throws InvalidIDException { - final ParsedObjectID parsedID = ParsedObjectID.parsedStringifiedObjectID(value,UUID.PROTOCOL); - return UUID.stringToObject(parsedID.getRemainder()); - } - - /** - * Return the name of the protocol that this factory uses. - * @return the protocol name - */ - public String getProtocol() { - return UUID.PROTOCOL; - } - - // ------------------------------------------------------------------------- - // G E N E R A T I O N M E T H O D S - // ------------------------------------------------------------------------- - - /** - * Create a new ObjectID instance using this protocol. - * @return Universally unique ID (UUID) - */ - public ObjectID create() { - return new UUID(java.util.UUID.randomUUID()); - } - -} - Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/CapabilitiesConverter.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -99,6 +99,9 @@ tgtCaps.setCapabilitySupport(Capability.QUERY_ORDERBY_NULL_ORDERING, srcCaps.supportsOrderByNullOrdering()); tgtCaps.setCapabilitySupport(Capability.INSERT_WITH_ITERATOR, srcCaps.supportsInsertWithIterator()); tgtCaps.setCapabilitySupport(Capability.COMMON_TABLE_EXPRESSIONS, srcCaps.supportsCommonTableExpressions()); + tgtCaps.setCapabilitySupport(Capability.ADVANCED_OLAP, srcCaps.supportsAdvancedOlapOperations()); + tgtCaps.setCapabilitySupport(Capability.ELEMENTARY_OLAP, srcCaps.supportsAdvancedOlapOperations()); + tgtCaps.setCapabilitySupport(Capability.QUERY_AGGREGATES_ARRAY, srcCaps.supportsArrayAgg()); List functions = srcCaps.getSupportedFunctions(); if(functions != null && functions.size() > 0) { Iterator iter = functions.iterator(); Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/datamgr/LanguageBridgeFactory.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -570,10 +570,12 @@ } AggregateFunction translate(AggregateSymbol symbol) { - return new AggregateFunction(symbol.getAggregateFunction().name(), + AggregateFunction af = new AggregateFunction(symbol.getAggregateFunction().name(), symbol.isDistinct(), translate(symbol.getExpression()), symbol.getType()); + af.setCondition(translate(symbol.getCondition())); + return af; } org.teiid.language.Expression translate(ExpressionSymbol symbol) { Modified: trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java =================================================================== --- trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/dqp/internal/process/Request.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -43,7 +43,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.core.id.IDGenerator; -import org.teiid.core.id.IntegerIDFactory; import org.teiid.core.types.DataTypeManager; import org.teiid.core.util.Assertion; import org.teiid.dqp.internal.datamgr.ConnectorManagerRepository; @@ -150,7 +149,6 @@ this.processorDataManager = processorDataManager; this.transactionService = transactionService; this.tempTableStore = tempTableStore; - idGenerator.setDefaultFactory(new IntegerIDFactory()); this.workContext = workContext; this.requestId = workContext.getRequestID(this.requestMsg.getExecutionId()); this.connectorManagerRepo = workContext.getVDB().getAttachment(ConnectorManagerRepository.class); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/BatchedUpdatePlanner.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -30,7 +30,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidRuntimeException; import org.teiid.core.id.IDGenerator; -import org.teiid.core.id.IntegerID; import org.teiid.query.QueryPlugin; import org.teiid.query.analysis.AnalysisRecord; import org.teiid.query.metadata.QueryMetadataInterface; @@ -142,9 +141,9 @@ } // If two or more contiguous commands made on the same model were found, then batch them if (batch.size() > 1) { - ProjectNode projectNode = new ProjectNode(((IntegerID)idGenerator.create()).getValue()); + ProjectNode projectNode = new ProjectNode(idGenerator.nextInt()); // Create a BatchedUpdateNode that creates a batched request for the connector - BatchedUpdateNode batchNode = new BatchedUpdateNode(((IntegerID)idGenerator.create()).getValue(), + BatchedUpdateNode batchNode = new BatchedUpdateNode(idGenerator.nextInt(), batch, contexts, shouldEvaluate, modelName); List symbols = batchedUpdateCommand.getProjectedSymbols(); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/QueryOptimizer.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -34,7 +34,6 @@ import org.teiid.core.TeiidProcessingException; import org.teiid.core.TeiidRuntimeException; import org.teiid.core.id.IDGenerator; -import org.teiid.core.id.IntegerIDFactory; import org.teiid.dqp.internal.process.PreparedPlan; import org.teiid.metadata.Procedure; import org.teiid.metadata.FunctionMethod.Determinism; @@ -99,7 +98,6 @@ // Create an ID generator that can be used for all plans to generate unique data node IDs if(idGenerator == null) { idGenerator = new IDGenerator(); - idGenerator.setDefaultFactory(new IntegerIDFactory()); } if(debug) { Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/capabilities/SourceCapabilities.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -321,6 +321,9 @@ INSERT_WITH_ITERATOR, COMMON_TABLE_EXPRESSIONS, MAX_DEPENDENT_PREDICATES, + ADVANCED_OLAP, + QUERY_AGGREGATES_ARRAY, + ELEMENTARY_OLAP } public enum Scope { Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/PlanToProcessConverter.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -32,8 +32,6 @@ import org.teiid.core.CoreConstants; import org.teiid.core.TeiidComponentException; import org.teiid.core.id.IDGenerator; -import org.teiid.core.id.IntegerID; -import org.teiid.core.id.IntegerIDFactory; import org.teiid.core.util.Assertion; import org.teiid.query.QueryPlugin; import org.teiid.query.analysis.AnalysisRecord; @@ -53,6 +51,7 @@ import org.teiid.query.processor.relational.DependentAccessNode; import org.teiid.query.processor.relational.DependentProcedureAccessNode; import org.teiid.query.processor.relational.DependentProcedureExecutionNode; +import org.teiid.query.processor.relational.EnhancedSortMergeJoinStrategy; import org.teiid.query.processor.relational.GroupingNode; import org.teiid.query.processor.relational.InsertPlanExecutionNode; import org.teiid.query.processor.relational.JoinNode; @@ -61,7 +60,6 @@ import org.teiid.query.processor.relational.NestedLoopJoinStrategy; import org.teiid.query.processor.relational.NestedTableJoinStrategy; import org.teiid.query.processor.relational.NullNode; -import org.teiid.query.processor.relational.EnhancedSortMergeJoinStrategy; import org.teiid.query.processor.relational.PlanExecutionNode; import org.teiid.query.processor.relational.ProjectIntoNode; import org.teiid.query.processor.relational.ProjectNode; @@ -169,8 +167,7 @@ } protected int getID() { - IntegerIDFactory intFactory = (IntegerIDFactory) idGenerator.getDefaultFactory(); - return ((IntegerID) intFactory.create()).getValue(); + return idGenerator.nextInt(); } protected RelationalNode convertNode(PlanNode node) @@ -377,7 +374,8 @@ break; case NodeConstants.Types.GROUP: GroupingNode gnode = new GroupingNode(getID()); - gnode.setGroupingElements( (List) node.getProperty(NodeConstants.Info.GROUP_COLS) ); + gnode.setGroupingElements( (List<Expression>) node.getProperty(NodeConstants.Info.GROUP_COLS) ); + gnode.setOutputMapping((SymbolMap)node.getProperty(NodeConstants.Info.SYMBOL_MAP)); gnode.setRemoveDuplicates(node.hasBooleanProperty(NodeConstants.Info.IS_DUP_REMOVAL)); processNode = gnode; break; Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -27,6 +27,8 @@ import java.util.Collection; import java.util.HashMap; import java.util.HashSet; +import java.util.Iterator; +import java.util.LinkedList; import java.util.List; import java.util.Map; import java.util.Set; @@ -45,9 +47,11 @@ import org.teiid.query.QueryPlugin; import org.teiid.query.analysis.AnalysisRecord; import org.teiid.query.mapping.relational.QueryNode; +import org.teiid.query.metadata.BasicQueryMetadata; import org.teiid.query.metadata.QueryMetadataInterface; import org.teiid.query.metadata.TempMetadataAdapter; import org.teiid.query.metadata.TempMetadataID; +import org.teiid.query.metadata.TempMetadataStore; import org.teiid.query.optimizer.QueryOptimizer; import org.teiid.query.optimizer.TriggerActionPlanner; import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; @@ -62,6 +66,8 @@ import org.teiid.query.optimizer.relational.rules.RuleCollapseSource; import org.teiid.query.optimizer.relational.rules.RuleConstants; import org.teiid.query.optimizer.relational.rules.RuleMergeCriteria; +import org.teiid.query.optimizer.relational.rules.RulePlaceAccess; +import org.teiid.query.optimizer.relational.rules.RulePushAggregates; import org.teiid.query.processor.ProcessorPlan; import org.teiid.query.processor.relational.AccessNode; import org.teiid.query.processor.relational.RelationalPlan; @@ -104,9 +110,13 @@ import org.teiid.query.sql.navigator.PreOrPostOrderNavigator; import org.teiid.query.sql.proc.CreateUpdateProcedureCommand; import org.teiid.query.sql.proc.TriggerAction; +import org.teiid.query.sql.symbol.AggregateSymbol; +import org.teiid.query.sql.symbol.AliasSymbol; import org.teiid.query.sql.symbol.AllSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; +import org.teiid.query.sql.symbol.Expression; +import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; @@ -115,6 +125,8 @@ import org.teiid.query.sql.util.SymbolMap; import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; import org.teiid.query.sql.visitor.CorrelatedReferenceCollectorVisitor; +import org.teiid.query.sql.visitor.ElementCollectorVisitor; +import org.teiid.query.sql.visitor.ExpressionMappingVisitor; import org.teiid.query.sql.visitor.GroupsUsedByElementsVisitor; import org.teiid.query.sql.visitor.ValueIteratorProviderCollectorVisitor; import org.teiid.query.util.CommandContext; @@ -401,7 +413,7 @@ rules.push(RuleConstants.PLAN_PROCEDURES); } if(hints.hasAggregates) { - rules.push(RuleConstants.PUSH_AGGREGATES); + rules.push(new RulePushAggregates(idGenerator)); } if(hints.hasJoin) { rules.push(RuleConstants.CHOOSE_DEPENDENT); @@ -729,9 +741,18 @@ } // Attach grouping node on top - if(query.hasAggregates()) { - plan = attachGrouping(plan, query, hints); + Collection<AggregateSymbol> aggs = AggregateSymbolCollectorVisitor.getAggregates(query.getSelect(), true); + boolean hasGrouping = !aggs.isEmpty(); + if (query.getHaving() != null) { + AggregateSymbolCollectorVisitor.getAggregates(query.getHaving(), aggs, null); + hasGrouping = true; } + if (query.getGroupBy() != null) { + hasGrouping = true; + } + if(hasGrouping) { + plan = attachGrouping(plan, query, aggs); + } // Attach having criteria node on top if(query.getHaving() != null) { @@ -982,7 +1003,7 @@ public static PlanNode createSelectNode(final Criteria crit, boolean isHaving) { PlanNode critNode = NodeFactory.getNewNode(NodeConstants.Types.SELECT); critNode.setProperty(NodeConstants.Info.SELECT_CRITERIA, crit); - if (isHaving && !AggregateSymbolCollectorVisitor.getAggregates(crit, false).isEmpty()) { + if (isHaving && !ElementCollectorVisitor.getAggregates(crit, false).isEmpty()) { critNode.setProperty(NodeConstants.Info.IS_HAVING, Boolean.TRUE); } // Add groups to crit node @@ -994,19 +1015,29 @@ /** * Attach a grouping node at top of tree. * @param plan Existing plan + * @param aggs * @param groupBy Group by clause, which may be null * @return Updated plan + * @throws TeiidComponentException + * @throws QueryMetadataException */ - private static PlanNode attachGrouping(PlanNode plan, Query query, PlanHints hints) { - PlanNode groupNode = NodeFactory.getNewNode(NodeConstants.Types.GROUP); - + private PlanNode attachGrouping(PlanNode plan, Query query, Collection<AggregateSymbol> aggs) throws QueryMetadataException, TeiidComponentException { + //TODO: correlated agg + GroupBy groupBy = query.getGroupBy(); - if(groupBy != null) { - groupNode.setProperty(NodeConstants.Info.GROUP_COLS, groupBy.getSymbols()); - groupNode.addGroups(GroupsUsedByElementsVisitor.getGroups(groupBy)); + List<Expression> groupingCols = null; + if (groupBy != null) { + groupingCols = groupBy.getSymbols(); } + + PlanNode groupNode = NodeFactory.getNewNode(NodeConstants.Types.GROUP); + + Map<Expression, ElementSymbol> mapping = buildGroupingNode(aggs, groupingCols, groupNode, this.context, this.idGenerator).inserseMapping(); attachLast(groupNode, plan); + ExpressionMappingVisitor.mapExpressions(query.getHaving(), mapping); + ExpressionMappingVisitor.mapExpressions(query.getSelect(), mapping); + ExpressionMappingVisitor.mapExpressions(query.getOrderBy(), mapping); // Mark in hints hints.hasAggregates = true; @@ -1014,6 +1045,57 @@ return groupNode; } + /** + * Build a grouping node that introduces a anon group (without a inline view source node) + */ + public static SymbolMap buildGroupingNode( + Collection<AggregateSymbol> aggs, List<? extends Expression> groupingCols, + PlanNode groupNode, CommandContext cc, IDGenerator idGenerator) throws QueryMetadataException, TeiidComponentException { + SymbolMap map = new SymbolMap(); + aggs = LanguageObject.Util.deepClone(aggs, AggregateSymbol.class); + groupingCols = LanguageObject.Util.deepClone(groupingCols, Expression.class); + GroupSymbol group = new GroupSymbol("anon_grp" + idGenerator.nextInt()); //$NON-NLS-1$ + if (!cc.getGroups().add(group.getName())) { + group = RulePlaceAccess.recontextSymbol(group, cc.getGroups()); + } + + TempMetadataStore tms = new TempMetadataStore(); + + int i = 0; + + List<AliasSymbol> symbols = new LinkedList<AliasSymbol>(); + List<Expression> targets = new LinkedList<Expression>(); + + if(groupingCols != null) { + groupNode.setProperty(NodeConstants.Info.GROUP_COLS, groupingCols); + groupNode.addGroups(GroupsUsedByElementsVisitor.getGroups(groupingCols)); + for (Expression ex : groupingCols) { + AliasSymbol as = new AliasSymbol("gcol" + i++, new ExpressionSymbol("expr", ex)); //$NON-NLS-1$ //$NON-NLS-2$ + targets.add(ex); + symbols.add(as); + } + } + + i = 0; + for (AggregateSymbol ex : aggs) { + AliasSymbol as = new AliasSymbol("agg" + i++, new ExpressionSymbol("expr", ex)); //$NON-NLS-1$ //$NON-NLS-2$ + targets.add(ex); + symbols.add(as); + } + + group.setMetadataID(tms.addTempGroup(group.getName(), symbols, true, false)); + Iterator<Expression> targetIter = targets.iterator(); + for (ElementSymbol es : ResolverUtil.resolveElementsInGroup(group, new TempMetadataAdapter(new BasicQueryMetadata(), tms))) { + Expression target = targetIter.next(); + es.setAggregate(target instanceof AggregateSymbol); + map.addMapping(es, target); + } + + groupNode.setProperty(NodeConstants.Info.SYMBOL_MAP, map); + groupNode.addGroup(group); + return map; + } + /** * Attach SORT node at top of tree. The SORT may be pushed down to a source (or sources) * if possible by the optimizer. Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/plantree/NodeConstants.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/plantree/NodeConstants.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/plantree/NodeConstants.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -130,7 +130,7 @@ MAKE_IND, // Group node properties - GROUP_COLS, // List <SingleElementSymbol> + GROUP_COLS, // List <Expression> // Special constant used in converting plan to process for all nodes OUTPUT_COLS, // List <SingleElementSymbol> Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/CapabilitiesUtil.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/CapabilitiesUtil.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/CapabilitiesUtil.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -120,12 +120,13 @@ return false; } boolean supportsFunctionsInGroupBy = caps.supportsCapability(Capability.QUERY_FUNCTIONS_IN_GROUP_BY); + boolean supportsInlineView = caps.supportsCapability(Capability.QUERY_FROM_INLINE_VIEWS); // Also verify that if there is a function that we can support pushdown of functions in group by Iterator colIter = groupCols.iterator(); while(colIter.hasNext()) { Expression col = (Expression) colIter.next(); - if(!(col instanceof ElementSymbol) && !supportsFunctionsInGroupBy) { + if(!(col instanceof ElementSymbol) && !supportsFunctionsInGroupBy && !supportsInlineView) { // Function in GROUP BY can't be pushed return false; } @@ -179,6 +180,11 @@ return false; } break; + case ARRAY_AGG: + if(! caps.supportsCapability(Capability.QUERY_AGGREGATES_ARRAY)) { + return false; + } + break; default: if (aggregate.isEnhancedNumeric()) { if (!caps.supportsCapability(Capability.QUERY_AGGREGATES_ENHANCED_NUMERIC)) { @@ -195,10 +201,12 @@ return false; } + if (aggregate.getCondition() != null && !caps.supportsCapability(Capability.ADVANCED_OLAP)) { + return false; + } + // Passed all the checks! return true; - - } public static boolean supportsScalarFunction(Object modelID, Function function, QueryMetadataInterface metadata, CapabilitiesFinder capFinder) Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/FrameUtil.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/FrameUtil.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/FrameUtil.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -45,6 +45,7 @@ import org.teiid.query.optimizer.relational.plantree.NodeEditor; import org.teiid.query.optimizer.relational.plantree.NodeFactory; import org.teiid.query.optimizer.relational.plantree.PlanNode; +import org.teiid.query.optimizer.relational.plantree.NodeConstants.Info; import org.teiid.query.processor.ProcessorPlan; import org.teiid.query.resolver.util.AccessPattern; import org.teiid.query.resolver.util.ResolverUtil; @@ -178,6 +179,10 @@ // multiple new groups. static void convertNode(PlanNode node, GroupSymbol oldGroup, Set<GroupSymbol> newGroups, Map symbolMap, QueryMetadataInterface metadata, boolean rewrite) throws QueryPlannerException { + + if (node.getType() == NodeConstants.Types.GROUP) { + correctSymbolMap(symbolMap, node); + } // Convert expressions from correlated subquery references; List<SymbolMap> refMaps = node.getAllReferences(); @@ -256,7 +261,7 @@ GroupsUsedByElementsVisitor.getGroups(orderBy, groups); } } else if(type == NodeConstants.Types.GROUP) { - List<SingleElementSymbol> groupCols = (List<SingleElementSymbol>)node.getProperty(NodeConstants.Info.GROUP_COLS); + List<Expression> groupCols = (List<Expression>)node.getProperty(NodeConstants.Info.GROUP_COLS); if (groupCols != null) { GroupBy groupBy= new GroupBy(groupCols); ExpressionMappingVisitor.mapExpressions(groupBy, symbolMap); @@ -265,6 +270,10 @@ GroupsUsedByElementsVisitor.getGroups(groupBy, groups); } } + if (!singleMapping) { + //add back the anon group + groups.add(((SymbolMap)node.getProperty(Info.SYMBOL_MAP)).asMap().keySet().iterator().next().getGroupSymbol()); + } } else if (type == NodeConstants.Types.SOURCE || type == NodeConstants.Types.ACCESS) { convertAccessPatterns(symbolMap, node); } @@ -281,7 +290,9 @@ if (mappedSymbol != null) { return mappedSymbol; } - return expression; + if (expression instanceof ElementSymbol) { + return expression; + } } ExpressionMappingVisitor.mapExpressions(expression, symbolMap); @@ -377,8 +388,7 @@ private static PlanNode findOriginatingNode(PlanNode root, Set<GroupSymbol> groups, boolean joinSource) { boolean containsGroups = false; - if(root.getType() == NodeConstants.Types.NULL || root.getType() == NodeConstants.Types.SOURCE - || root.getType() == NodeConstants.Types.JOIN || root.getType() == NodeConstants.Types.SET_OP || + if((root.getType() & (NodeConstants.Types.NULL | NodeConstants.Types.SOURCE | NodeConstants.Types.JOIN | NodeConstants.Types.SET_OP | NodeConstants.Types.GROUP)) == root.getType() || (joinSource && root.getType() == NodeConstants.Types.ACCESS)) { //if there are no groups then the first possible match is the one we want Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/NewCalculateCostUtil.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/NewCalculateCostUtil.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/NewCalculateCostUtil.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -74,7 +74,6 @@ import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.util.SymbolMap; -import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; import org.teiid.query.sql.visitor.ElementCollectorVisitor; import org.teiid.query.sql.visitor.EvaluatableVisitor; import org.teiid.query.sql.visitor.GroupsUsedByElementsVisitor; @@ -440,7 +439,7 @@ if (stats == null) { if (node.getType() == NodeConstants.Types.PROJECT) { Collection<SingleElementSymbol> elems = new HashSet<SingleElementSymbol>(); - AggregateSymbolCollectorVisitor.getAggregates(expr, elems, elems); + ElementCollectorVisitor.getElements(expr, elems); newStats[Stat.NDV.ordinal()] = getStat(Stat.NDV, elems, node, childCardinality, metadata); newStats[Stat.NNV.ordinal()] = getStat(Stat.NNV, elems, node, childCardinality, metadata); } else { @@ -578,9 +577,8 @@ if (node.getType() == NodeConstants.Types.PROJECT) { return (List<? extends Expression>) node.getProperty(NodeConstants.Info.PROJECT_COLS); } else if (node.getType() == NodeConstants.Types.GROUP) { - LinkedList<Expression> result = new LinkedList<Expression>(RulePushAggregates.collectAggregates(node)); - result.addAll((Collection<? extends Expression>) node.getProperty(Info.GROUP_COLS)); - return result; + SymbolMap map = (SymbolMap)node.getProperty(Info.SYMBOL_MAP); + return map.getKeys(); } LinkedList<ElementSymbol> elements = new LinkedList<ElementSymbol>(); for (GroupSymbol group : node.getGroups()) { @@ -1014,7 +1012,7 @@ return cost; } - static boolean usesKey(PlanNode planNode, Collection<? extends SingleElementSymbol> allElements, QueryMetadataInterface metadata) throws QueryMetadataException, TeiidComponentException { + static boolean usesKey(PlanNode planNode, Collection<? extends Expression> allElements, QueryMetadataInterface metadata) throws QueryMetadataException, TeiidComponentException { //TODO: key preserved joins should be marked return isSingleTable(planNode) && usesKey(allElements, metadata); @@ -1035,17 +1033,17 @@ * we are in the plan. * if a key column is used after a non 1-1 join or a union all, then it may be non-unique. */ - public static boolean usesKey(Collection<? extends SingleElementSymbol> allElements, QueryMetadataInterface metadata) + public static boolean usesKey(Collection<? extends Expression> allElements, QueryMetadataInterface metadata) throws QueryMetadataException, TeiidComponentException { return usesKey(allElements, null, metadata, true); } - public static boolean usesKey(Collection<? extends SingleElementSymbol> allElements, Set<GroupSymbol> groups, QueryMetadataInterface metadata, boolean unique) + public static boolean usesKey(Collection<? extends Expression> allElements, Set<GroupSymbol> groups, QueryMetadataInterface metadata, boolean unique) throws QueryMetadataException, TeiidComponentException { return getKeyUsed(allElements, groups, metadata, unique) != null; } - public static Object getKeyUsed(Collection<? extends SingleElementSymbol> allElements, Set<GroupSymbol> groups, QueryMetadataInterface metadata, Boolean unique) + public static Object getKeyUsed(Collection<? extends Expression> allElements, Set<GroupSymbol> groups, QueryMetadataInterface metadata, Boolean unique) throws QueryMetadataException, TeiidComponentException { if(allElements == null || allElements.size() == 0) { @@ -1054,7 +1052,7 @@ // Sort elements into groups Map<GroupSymbol, List<Object>> groupMap = new HashMap<GroupSymbol, List<Object>>(); - for (SingleElementSymbol ses : allElements) { + for (Expression ses : allElements) { Expression ex = SymbolMap.getExpression(ses); if (!(ex instanceof ElementSymbol)) { continue; //TODO: function based indexes are possible, but we don't have the metadata @@ -1337,7 +1335,7 @@ static float getNDVEstimate(PlanNode indNode, QueryMetadataInterface metadata, float cardinality, - Collection<? extends SingleElementSymbol> elems, boolean useCardinalityIfUnknown) throws QueryMetadataException, + Collection<? extends Expression> elems, boolean useCardinalityIfUnknown) throws QueryMetadataException, TeiidComponentException { if (elems == null || elems.isEmpty()) { return cardinality; Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleAssignOutputElements.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -57,7 +57,7 @@ import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.util.SymbolMap; -import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; +import org.teiid.query.sql.visitor.ElementCollectorVisitor; import org.teiid.query.sql.visitor.GroupsUsedByElementsVisitor; import org.teiid.query.util.CommandContext; @@ -437,8 +437,6 @@ * are any symbols that are required in the processing of this node, * for instance to create a new element symbol or sort on it, etc. * @param node Node to collect for - * @param requiredSymbols Collection<SingleElementSymbol> Place to collect required symbols - * @param createdSymbols Set<SingleElementSymbol> Place to collect any symbols created by this node */ private List<SingleElementSymbol> collectRequiredInputSymbols(PlanNode node) { @@ -459,57 +457,53 @@ } if (ss instanceof ExpressionSymbol && !(ss instanceof AggregateSymbol)) { - ExpressionSymbol exprSymbol = (ExpressionSymbol)ss; - - if (!exprSymbol.isDerivedExpression()) { - createdSymbols.add(ss); - } + createdSymbols.add(ss); } - AggregateSymbolCollectorVisitor.getAggregates(ss, requiredSymbols, requiredSymbols); + ElementCollectorVisitor.getElements(ss, requiredSymbols); } break; } case NodeConstants.Types.SELECT: Criteria selectCriteria = (Criteria) node.getProperty(NodeConstants.Info.SELECT_CRITERIA); - AggregateSymbolCollectorVisitor.getAggregates(selectCriteria, requiredSymbols, requiredSymbols); + ElementCollectorVisitor.getElements(selectCriteria, requiredSymbols); break; case NodeConstants.Types.JOIN: List<Criteria> crits = (List) node.getProperty(NodeConstants.Info.JOIN_CRITERIA); if(crits != null) { for (Criteria joinCriteria : crits) { - AggregateSymbolCollectorVisitor.getAggregates(joinCriteria, requiredSymbols, requiredSymbols); + ElementCollectorVisitor.getElements(joinCriteria, requiredSymbols); } } break; case NodeConstants.Types.GROUP: - List<SingleElementSymbol> groupCols = (List<SingleElementSymbol>) node.getProperty(NodeConstants.Info.GROUP_COLS); + List<Expression> groupCols = (List<Expression>) node.getProperty(NodeConstants.Info.GROUP_COLS); if(groupCols != null) { - for (SingleElementSymbol expression : groupCols) { - if(expression instanceof ElementSymbol || expression instanceof AggregateSymbol) { - requiredSymbols.add(expression); - } else { - ExpressionSymbol exprSymbol = (ExpressionSymbol) expression; - Expression expr = exprSymbol.getExpression(); - AggregateSymbolCollectorVisitor.getAggregates(expr, requiredSymbols, requiredSymbols); - createdSymbols.add(exprSymbol); - } + for (Expression expression : groupCols) { + ElementCollectorVisitor.getElements(expression, requiredSymbols); } } - + + SymbolMap symbolMap = (SymbolMap) node.getProperty(NodeConstants.Info.SYMBOL_MAP); + // Take credit for creating any aggregates that are needed above for (SingleElementSymbol outputSymbol : outputCols) { - if(outputSymbol instanceof AggregateSymbol) { - AggregateSymbol agg = (AggregateSymbol)outputSymbol; - createdSymbols.add(outputSymbol); + createdSymbols.add(outputSymbol); + Expression ex = symbolMap.getMappedExpression((ElementSymbol) outputSymbol); + if(ex instanceof AggregateSymbol) { + AggregateSymbol agg = (AggregateSymbol)ex; Expression aggExpr = agg.getExpression(); if(aggExpr != null) { - AggregateSymbolCollectorVisitor.getAggregates(aggExpr, requiredSymbols, requiredSymbols); + ElementCollectorVisitor.getElements(aggExpr, requiredSymbols); } OrderBy orderBy = agg.getOrderBy(); if(orderBy != null) { - AggregateSymbolCollectorVisitor.getAggregates(orderBy, requiredSymbols, requiredSymbols); + ElementCollectorVisitor.getElements(orderBy, requiredSymbols); } + Expression condition = agg.getCondition(); + if(condition != null) { + ElementCollectorVisitor.getElements(condition, requiredSymbols); + } } } @@ -519,7 +513,7 @@ // Gather elements from correlated subquery references; for (SymbolMap refs : node.getAllReferences()) { for (Expression expr : refs.asMap().values()) { - AggregateSymbolCollectorVisitor.getAggregates(expr, requiredSymbols, requiredSymbols); + ElementCollectorVisitor.getElements(expr, requiredSymbols); } } Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleChooseJoinStrategy.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -71,7 +71,6 @@ * Determines whether this node should be converted to a merge join node * @param joinNode The join node * @param metadata The metadata - * @return True if merge is possible */ static void chooseJoinStrategy(PlanNode joinNode, QueryMetadataInterface metadata) { // Check that join is an inner join @@ -183,7 +182,6 @@ ExpressionSymbol expressionSymbol = uniqueExpressions.get(expression); if (expressionSymbol == null) { expressionSymbol = new ExpressionSymbol("$" + EXPRESSION_INDEX.getAndIncrement(), expression); //$NON-NLS-1$ - expressionSymbol.setDerivedExpression(true); uniqueExpressions.put(expression, expressionSymbol); } result.add(expressionSymbol); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleCollapseSource.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleCollapseSource.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleCollapseSource.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -24,14 +24,19 @@ import java.util.ArrayList; import java.util.Collection; +import java.util.HashMap; import java.util.HashSet; +import java.util.Iterator; import java.util.LinkedHashSet; import java.util.LinkedList; import java.util.List; +import java.util.Set; import org.teiid.api.exception.query.QueryMetadataException; import org.teiid.api.exception.query.QueryPlannerException; import org.teiid.core.TeiidComponentException; +import org.teiid.core.TeiidException; +import org.teiid.core.TeiidRuntimeException; import org.teiid.core.types.DataTypeManager; import org.teiid.query.analysis.AnalysisRecord; import org.teiid.query.metadata.QueryMetadataInterface; @@ -72,6 +77,7 @@ import org.teiid.query.sql.lang.UnaryFromClause; import org.teiid.query.sql.lang.SetQuery.Operation; import org.teiid.query.sql.navigator.DeepPostOrderNavigator; +import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; @@ -80,6 +86,7 @@ import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.util.SymbolMap; +import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; import org.teiid.query.sql.visitor.ExpressionMappingVisitor; import org.teiid.query.sql.visitor.ValueIteratorProviderCollectorVisitor; import org.teiid.query.util.CommandContext; @@ -241,11 +248,12 @@ if (query.getCriteria() instanceof CompoundCriteria) { query.setCriteria(QueryRewriter.optimizeCriteria((CompoundCriteria)query.getCriteria(), metadata)); } - if (!CapabilitiesUtil.useAnsiJoin(RuleRaiseAccess.getModelIDFromAccess(accessRoot, metadata), metadata, capFinder)) { + Object modelID = RuleRaiseAccess.getModelIDFromAccess(accessRoot, metadata); + if (!CapabilitiesUtil.useAnsiJoin(modelID, metadata, capFinder)) { simplifyFromClause(query); } if (columns.isEmpty()) { - if (CapabilitiesUtil.supports(Capability.QUERY_SELECT_EXPRESSION, RuleRaiseAccess.getModelIDFromAccess(accessRoot, metadata), metadata, capFinder)) { + if (CapabilitiesUtil.supports(Capability.QUERY_SELECT_EXPRESSION, modelID, metadata, capFinder)) { select.addSymbol(new ExpressionSymbol("dummy", new Constant(1))); //$NON-NLS-1$ } else { //TODO: need to ensure the type is consistent @@ -253,6 +261,20 @@ select.addSymbol(selectOutputElement(query.getFrom().getGroups(), metadata)); } } + PlanNode groupNode = NodeEditor.findNodePreOrder(node, NodeConstants.Types.GROUP, NodeConstants.Types.SOURCE); + if (groupNode != null) { + SymbolMap symbolMap = (SymbolMap) groupNode.getProperty(NodeConstants.Info.SYMBOL_MAP); + + //map back to expression form + ExpressionMappingVisitor.mapExpressions(query.getOrderBy(), symbolMap.asMap()); + ExpressionMappingVisitor.mapExpressions(query.getSelect(), symbolMap.asMap()); + ExpressionMappingVisitor.mapExpressions(query.getHaving(), symbolMap.asMap()); + + if (!CapabilitiesUtil.supports(Capability.QUERY_FUNCTIONS_IN_GROUP_BY, modelID, metadata, capFinder)) { + //if group by expressions are not support, add an inline view to compensate + query = RuleCollapseSource.rewriteGroupByExpressionsAsView(query, metadata); + } + } return query; } @@ -273,7 +295,7 @@ for (ElementSymbol element : elements) { if(metadata.elementSupports(element.getMetadataID(), SupportConstants.Element.SELECT)) { - element = (ElementSymbol)element.clone(); + element = element.clone(); element.setGroupSymbol(group); return element; } @@ -323,10 +345,10 @@ // Get last two clauses added to the FROM and combine them into a JoinPredicate From from = query.getFrom(); - List clauses = from.getClauses(); + List<FromClause> clauses = from.getClauses(); int lastClause = clauses.size()-1; - FromClause clause1 = (FromClause) clauses.get(lastClause-1); - FromClause clause2 = (FromClause) clauses.get(lastClause); + FromClause clause1 = clauses.get(lastClause-1); + FromClause clause2 = clauses.get(lastClause); //correct the criteria or the join type if necessary if (joinType != JoinType.JOIN_CROSS && crits.isEmpty()) { @@ -492,13 +514,13 @@ */ private void simplifyFromClause(Query query) { From from = query.getFrom(); - List clauses = from.getClauses(); - FromClause rootClause = (FromClause) clauses.get(0); + List<FromClause> clauses = from.getClauses(); + FromClause rootClause = clauses.get(0); // If all joins are inner joins, move criteria to WHERE and make // FROM a list of groups instead of a tree of JoinPredicates if(! hasOuterJoins(rootClause)) { - from.setClauses(new ArrayList()); + from.setClauses(new ArrayList<FromClause>()); shredJoinTree(rootClause, query); } // else leave as is } @@ -554,4 +576,75 @@ return "CollapseSource"; //$NON-NLS-1$ } + public static Query rewriteGroupByExpressionsAsView(Query query, QueryMetadataInterface metadata) { + if (query.getGroupBy() == null) { + return query; + } + // we check for group by expressions here to create an ANSI SQL plan + boolean hasExpression = false; + for (final Iterator<Expression> iterator = query.getGroupBy().getSymbols().iterator(); !hasExpression && iterator.hasNext();) { + hasExpression = !(iterator.next() instanceof ElementSymbol); + } + if (!hasExpression) { + return query; + } + Select select = query.getSelect(); + GroupBy groupBy = query.getGroupBy(); + query.setGroupBy(null); + Criteria having = query.getHaving(); + query.setHaving(null); + OrderBy orderBy = query.getOrderBy(); + query.setOrderBy(null); + Limit limit = query.getLimit(); + query.setLimit(null); + Set<Expression> newSelectColumns = new HashSet<Expression>(); + for (final Iterator<Expression> iterator = groupBy.getSymbols().iterator(); iterator.hasNext();) { + newSelectColumns.add(iterator.next()); + } + Set<AggregateSymbol> aggs = new HashSet<AggregateSymbol>(); + aggs.addAll(AggregateSymbolCollectorVisitor.getAggregates(select, true)); + if (having != null) { + aggs.addAll(AggregateSymbolCollectorVisitor.getAggregates(having, true)); + } + for (AggregateSymbol aggregateSymbol : aggs) { + if (aggregateSymbol.getExpression() != null) { + Expression expr = aggregateSymbol.getExpression(); + newSelectColumns.add(SymbolMap.getExpression(expr)); + } + } + Select innerSelect = new Select(); + int index = 0; + for (Expression expr : newSelectColumns) { + if (expr instanceof SingleElementSymbol) { + innerSelect.addSymbol((SingleElementSymbol)expr); + } else { + innerSelect.addSymbol(new ExpressionSymbol("EXPR" + index++ , expr)); //$NON-NLS-1$ + } + } + query.setSelect(innerSelect); + Query outerQuery = null; + try { + outerQuery = QueryRewriter.createInlineViewQuery(new GroupSymbol("X"), query, metadata, query.getSelect().getProjectedSymbols()); //$NON-NLS-1$ + } catch (TeiidException err) { + throw new TeiidRuntimeException(err); + } + Iterator<SingleElementSymbol> iter = outerQuery.getSelect().getProjectedSymbols().iterator(); + HashMap<Expression, SingleElementSymbol> expressionMap = new HashMap<Expression, SingleElementSymbol>(); + for (SingleElementSymbol symbol : query.getSelect().getProjectedSymbols()) { + expressionMap.put(SymbolMap.getExpression(symbol), iter.next()); + } + ExpressionMappingVisitor.mapExpressions(groupBy, expressionMap); + outerQuery.setGroupBy(groupBy); + ExpressionMappingVisitor.mapExpressions(having, expressionMap); + outerQuery.setHaving(having); + ExpressionMappingVisitor.mapExpressions(orderBy, expressionMap); + outerQuery.setOrderBy(orderBy); + outerQuery.setLimit(limit); + ExpressionMappingVisitor.mapExpressions(select, expressionMap); + outerQuery.setSelect(select); + outerQuery.setOption(query.getOption()); + query = outerQuery; + return query; + } + } Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleConstants.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleConstants.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleConstants.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -34,7 +34,6 @@ public static final OptimizerRule MERGE_VIRTUAL = new RuleMergeVirtual(); public static final OptimizerRule CHOOSE_JOIN_STRATEGY = new RuleChooseJoinStrategy(); public static final OptimizerRule RAISE_ACCESS = new RuleRaiseAccess(); - public static final OptimizerRule PUSH_AGGREGATES = new RulePushAggregates(); public static final OptimizerRule CHOOSE_DEPENDENT = new RuleChooseDependent(); public static final OptimizerRule COLLAPSE_SOURCE = new RuleCollapseSource(); public static final OptimizerRule ASSIGN_OUTPUT_ELEMENTS = new RuleAssignOutputElements(); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleDecomposeJoin.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleDecomposeJoin.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleDecomposeJoin.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -25,7 +25,6 @@ import java.util.ArrayList; import java.util.Collections; import java.util.HashMap; -import java.util.HashSet; import java.util.LinkedHashMap; import java.util.LinkedList; import java.util.List; @@ -208,20 +207,15 @@ throws TeiidComponentException, QueryPlannerException, QueryMetadataException { Set<String> groups = context.getGroups(); - if (groups == null) { - groups = new HashSet<String>(); - context.setGroups(groups); - } group = RulePlaceAccess.recontextSymbol(group, groups); PlanNode projectNode = NodeEditor.findNodePreOrder(newUnion, NodeConstants.Types.PROJECT); List<? extends SingleElementSymbol> projectedSymbols = (List<? extends SingleElementSymbol>)projectNode.getProperty(Info.PROJECT_COLS); - PlanNode view = RulePushAggregates.createView(group, projectedSymbols, newUnion, metadata); + SymbolMap newSymbolMap = RulePushAggregates.createSymbolMap(group, projectedSymbols, newUnion, metadata); + PlanNode view = RuleDecomposeJoin.createSource(group, newUnion, newSymbolMap); - SymbolMap newSymbolMap = (SymbolMap)view.getProperty(Info.SYMBOL_MAP); - Map<Expression, ElementSymbol> inverseMap = newSymbolMap.inserseMapping(); toReplace.getParent().replaceChild(toReplace, view); Set<GroupSymbol> newGroups = Collections.singleton(group); Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeCriteria.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -591,7 +591,7 @@ boolean distinct = false; if (query.getGroupBy() != null) { distinct = true; - for (SingleElementSymbol groupByExpr : (List<SingleElementSymbol>)query.getGroupBy().getSymbols()) { + for (Expression groupByExpr : query.getGroupBy().getSymbols()) { if (!expressions.contains(groupByExpr)) { distinct = false; break; Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeVirtual.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeVirtual.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleMergeVirtual.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -41,6 +41,7 @@ import org.teiid.query.optimizer.relational.plantree.NodeEditor; import org.teiid.query.optimizer.relational.plantree.PlanNode; import org.teiid.query.optimizer.relational.plantree.NodeConstants.Info; +import org.teiid.query.resolver.util.AccessPattern; import org.teiid.query.sql.lang.JoinType; import org.teiid.query.sql.lang.OrderBy; import org.teiid.query.sql.lang.OrderByItem; @@ -161,13 +162,7 @@ return root; } - PlanNode parentGroup = NodeEditor.findParent(frame, NodeConstants.Types.GROUP, NodeConstants.Types.SOURCE); - List<SingleElementSymbol> groupCols = null; - if (parentGroup != null) { - groupCols = (List<SingleElementSymbol>)parentGroup.getProperty(NodeConstants.Info.GROUP_COLS); - } - - if (!checkProjectedSymbols(projectNode, virtualGroup, parentJoin, groupCols, symbolMap, metadata)) { + if (!checkProjectedSymbols(projectNode, virtualGroup, parentJoin, metadata)) { return root; } @@ -190,9 +185,9 @@ // find the new root of the frame so that access patterns can be propagated PlanNode newRoot = FrameUtil.findJoinSourceNode(frame.getFirstChild()); if (newRoot != null) { - Collection ap = (Collection)frame.getProperty(NodeConstants.Info.ACCESS_PATTERNS); + Collection<AccessPattern> ap = (Collection)frame.getProperty(NodeConstants.Info.ACCESS_PATTERNS); if (ap != null) { - Collection newAp = (Collection)newRoot.getProperty(NodeConstants.Info.ACCESS_PATTERNS); + Collection<AccessPattern> newAp = (Collection)newRoot.getProperty(NodeConstants.Info.ACCESS_PATTERNS); if (newAp == null) { newRoot.setProperty(NodeConstants.Info.ACCESS_PATTERNS, ap); } else { @@ -323,8 +318,6 @@ private static boolean checkProjectedSymbols(PlanNode projectNode, GroupSymbol virtualGroup, PlanNode parentJoin, - List<SingleElementSymbol> parentGroupingCols, - SymbolMap symbolMap, QueryMetadataInterface metadata) { List<SingleElementSymbol> selectSymbols = (List<SingleElementSymbol>)projectNode.getProperty(NodeConstants.Info.PROJECT_COLS); @@ -351,7 +344,6 @@ } } - List<ElementSymbol> virtualElements = symbolMap.getKeys(); for (int i = 0; i < selectSymbols.size(); i++) { SingleElementSymbol symbol = selectSymbols.get(i); Collection scalarSubqueries = ValueIteratorProviderCollectorVisitor.getValueIteratorProviders(symbol); @@ -361,9 +353,6 @@ if (checkForNullDependent && JoinUtil.isNullDependent(metadata, groups, SymbolMap.getExpression(symbol))) { return false; } - if (parentGroupingCols != null && !(SymbolMap.getExpression(symbol) instanceof SingleElementSymbol) && parentGroupingCols.contains(virtualElements.get(i))) { - return false; - } // TEIID-16: We do not want to merge a non-deterministic scalar function if (FunctionCollectorVisitor.isNonDeterministic(symbol)) { return false; Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlaceAccess.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlaceAccess.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlaceAccess.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -69,10 +69,6 @@ QueryPlannerException { Set<String> groups = context.getGroups(); - if (groups == null) { - groups = new HashSet<String>(); - context.setGroups(groups); - } boolean[] addtionalRules = new boolean[2]; Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlanSorts.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlanSorts.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePlanSorts.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -22,6 +22,7 @@ package org.teiid.query.optimizer.relational.rules; +import java.util.ArrayList; import java.util.List; import org.teiid.api.exception.query.QueryMetadataException; @@ -43,8 +44,11 @@ import org.teiid.query.sql.lang.SetQuery; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.AliasSymbol; +import org.teiid.query.sql.symbol.ElementSymbol; +import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; +import org.teiid.query.sql.util.SymbolMap; import org.teiid.query.util.CommandContext; @@ -86,13 +90,25 @@ root = checkForProjectOptimization(node, root, metadata, capFinder, record); } List<SingleElementSymbol> orderColumns = ((OrderBy)node.getProperty(NodeConstants.Info.SORT_ORDER)).getSortKeys(); + List<Expression> sortExpressions = new ArrayList<Expression>(orderColumns.size()); PlanNode possibleSort = NodeEditor.findNodePreOrder(node, NodeConstants.Types.GROUP, NodeConstants.Types.SOURCE | NodeConstants.Types.ACCESS); if (possibleSort != null) { - List exprs = (List)possibleSort.getProperty(Info.GROUP_COLS); - if (exprs != null && exprs.containsAll(orderColumns)) { - exprs.removeAll(orderColumns); - orderColumns.addAll(exprs); - possibleSort.setProperty(Info.GROUP_COLS, orderColumns); + boolean otherExpression = false; + SymbolMap groupMap = (SymbolMap)possibleSort.getProperty(Info.SYMBOL_MAP); + for (SingleElementSymbol singleElementSymbol : orderColumns) { + Expression ex = SymbolMap.getExpression(singleElementSymbol); + if (ex instanceof ElementSymbol) { + sortExpressions.add(groupMap.getMappedExpression((ElementSymbol) ex)); + } else { + otherExpression = true; + break; + } + } + + List<Expression> exprs = (List<Expression>)possibleSort.getProperty(Info.GROUP_COLS); + if (!otherExpression && exprs != null && exprs.containsAll(sortExpressions)) { + exprs.removeAll(sortExpressions); + exprs.addAll(0, sortExpressions); if (node.getParent() == null) { root = node.getFirstChild(); root.removeFromParent(); @@ -105,54 +121,6 @@ } break; } -/* possibleSort = NodeEditor.findNodePreOrder(node, NodeConstants.Types.JOIN, NodeConstants.Types.SOURCE | NodeConstants.Types.ACCESS); - if (possibleSort == null) { - break; - } - boolean left = false; - if (possibleSort.getType() == NodeConstants.Types.JOIN) { - if (possibleSort.getProperty(NodeConstants.Info.JOIN_STRATEGY) != JoinStrategyType.MERGE - || possibleSort.getProperty(NodeConstants.Info.JOIN_TYPE) != JoinType.JOIN_INNER) { - break; - } - if (FrameUtil.findJoinSourceNode(possibleSort.getFirstChild()).getGroups().containsAll(node.getGroups()) - && possibleSort.getProperty(NodeConstants.Info.SORT_LEFT) == SortOption.SORT) { - left = true; - } else if (!FrameUtil.findJoinSourceNode(possibleSort.getLastChild()).getGroups().containsAll(node.getGroups()) - || possibleSort.getProperty(NodeConstants.Info.SORT_RIGHT) != SortOption.SORT) { - break; - } - } - List exprs = (List)possibleSort.getProperty(left?Info.LEFT_EXPRESSIONS:Info.RIGHT_EXPRESSIONS); - if (exprs != null && exprs.containsAll(orderColumns)) { - List<Integer> indexes = new ArrayList<Integer>(orderColumns.size()); - for (Expression expr : (List<Expression>)orderColumns) { - indexes.add(0, exprs.indexOf(expr)); - } - exprs.removeAll(orderColumns); - List newExprs = new ArrayList(orderColumns); - newExprs.addAll(exprs); - possibleSort.setProperty(left?Info.LEFT_EXPRESSIONS:Info.RIGHT_EXPRESSIONS, newExprs); - if (node.getParent() == null) { - root = node.getFirstChild(); - root.removeFromParent(); - node = root; - } else { - PlanNode nextNode = node.getFirstChild(); - NodeEditor.removeChildNode(node.getParent(), node); - node = nextNode; - } - exprs = (List)possibleSort.getProperty(left?Info.RIGHT_EXPRESSIONS:Info.LEFT_EXPRESSIONS); - List toRemove = new ArrayList(); - for (Integer index : indexes) { - Object o = exprs.get(index); - exprs.add(0, o); - toRemove.add(o); - } - exprs.subList(indexes.size(), exprs.size()).removeAll(toRemove); - possibleSort.setProperty(left?NodeConstants.Info.SORT_LEFT:NodeConstants.Info.SORT_RIGHT, SortOption.SORT_REQUIRED); - } -*/ break; case NodeConstants.Types.DUP_REMOVE: if (parentBlocking) { @@ -243,13 +211,8 @@ if(ss instanceof AliasSymbol) { ss = ((AliasSymbol)ss).getSymbol(); } - if (ss instanceof ExpressionSymbol && !(ss instanceof AggregateSymbol)) { - ExpressionSymbol exprSymbol = (ExpressionSymbol)ss; - - if (!exprSymbol.isDerivedExpression()) { - return root; //TODO: insert a new project node to handle this case - } + return root; //TODO: insert a new project node to handle this case } if (!childOutputCols.contains(ss)) { return root; Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushAggregates.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -40,17 +40,18 @@ import org.teiid.api.exception.query.QueryPlannerException; import org.teiid.api.exception.query.QueryResolverException; import org.teiid.core.TeiidComponentException; +import org.teiid.core.id.IDGenerator; import org.teiid.core.types.DataTypeManager; import org.teiid.language.SQLConstants.NonReserved; import org.teiid.query.analysis.AnalysisRecord; import org.teiid.query.function.FunctionLibrary; import org.teiid.query.metadata.QueryMetadataInterface; import org.teiid.query.metadata.TempMetadataAdapter; -import org.teiid.query.metadata.TempMetadataID; import org.teiid.query.metadata.TempMetadataStore; import org.teiid.query.optimizer.capabilities.CapabilitiesFinder; import org.teiid.query.optimizer.capabilities.SourceCapabilities.Capability; import org.teiid.query.optimizer.relational.OptimizerRule; +import org.teiid.query.optimizer.relational.RelationalPlanner; import org.teiid.query.optimizer.relational.RuleStack; import org.teiid.query.optimizer.relational.plantree.NodeConstants; import org.teiid.query.optimizer.relational.plantree.NodeEditor; @@ -61,7 +62,6 @@ import org.teiid.query.resolver.util.ResolverVisitor; import org.teiid.query.rewriter.QueryRewriter; import org.teiid.query.sql.LanguageObject; -import org.teiid.query.sql.LanguageObject.Util; import org.teiid.query.sql.lang.CompareCriteria; import org.teiid.query.sql.lang.Criteria; import org.teiid.query.sql.lang.IsNullCriteria; @@ -94,6 +94,12 @@ */ public class RulePushAggregates implements OptimizerRule { + + private IDGenerator idGenerator; + + public RulePushAggregates(IDGenerator idGenerator) { + this.idGenerator = idGenerator; + } /** * @see org.teiid.query.optimizer.relational.OptimizerRule#execute(org.teiid.query.optimizer.relational.plantree.PlanNode, @@ -113,13 +119,16 @@ for (PlanNode groupNode : NodeEditor.findAllNodes(plan, NodeConstants.Types.GROUP, NodeConstants.Types.ACCESS)) { PlanNode child = groupNode.getFirstChild(); - List<SingleElementSymbol> groupingExpressions = (List<SingleElementSymbol>)groupNode.getProperty(NodeConstants.Info.GROUP_COLS); + List<Expression> groupingExpressions = (List<Expression>)groupNode.getProperty(NodeConstants.Info.GROUP_COLS); + if (groupingExpressions == null) { + groupingExpressions = Collections.emptyList(); + } if (child.getType() == NodeConstants.Types.SOURCE) { PlanNode setOp = child.getFirstChild(); try { - pushGroupNodeOverUnion(plan, metadata, capFinder, groupNode, child, groupingExpressions, setOp, context, analysisRecord); + pushGroupNodeOverUnion(metadata, capFinder, groupNode, child, groupingExpressions, setOp, context, analysisRecord); } catch (QueryResolverException e) { throw new TeiidComponentException(e); } @@ -132,7 +141,7 @@ Set<AggregateSymbol> aggregates = collectAggregates(groupNode); - pushGroupNode(groupNode, groupingExpressions, aggregates, metadata, capFinder); + pushGroupNode(groupNode, groupingExpressions, aggregates, metadata, capFinder, context); } return plan; @@ -159,48 +168,16 @@ * ... * * Or if the child does not support pushdown we add dummy aggregate projection - * count(*) = 1, count(x) = case x is null then 0 else 1 end, avg(x) = x, etc. - * - * if partitioned, then we don't need decomposition or the top level group by - * - * source - * set op - * project - * [select] - * group [agg(x), {a, b}] - * source - * child 1 - * ... - * + * count(*) = 1, count(x) = case x is null then 0 else 1 end, avg(x) = x, etc. */ - private void pushGroupNodeOverUnion(PlanNode plan, - QueryMetadataInterface metadata, CapabilitiesFinder capFinder, + private void pushGroupNodeOverUnion(QueryMetadataInterface metadata, CapabilitiesFinder capFinder, PlanNode groupNode, PlanNode unionSourceParent, - List<SingleElementSymbol> groupingExpressions, PlanNode setOp, CommandContext context, AnalysisRecord record) + List<Expression> groupingExpressions, PlanNode setOp, CommandContext context, AnalysisRecord record) throws TeiidComponentException, QueryMetadataException, QueryPlannerException, QueryResolverException { - if (setOp == null) { + if (setOp == null || setOp.getProperty(NodeConstants.Info.SET_OPERATION) != Operation.UNION) { return; } - PlanNode intermediateView = null; - if (setOp.getType() != NodeConstants.Types.SET_OP) { - if (setOp.getType() != NodeConstants.Types.PROJECT) { - return; - } - intermediateView = unionSourceParent; - unionSourceParent = setOp.getFirstChild(); - if (unionSourceParent == null || unionSourceParent.getType() != NodeConstants.Types.SOURCE || unionSourceParent.getFirstChild() == null - || unionSourceParent.getFirstChild().getType() != NodeConstants.Types.SET_OP || unionSourceParent.getFirstChild().getProperty(NodeConstants.Info.SET_OPERATION) != Operation.UNION) { - return; //not an eligible union - } - setOp = unionSourceParent.getFirstChild(); - if (groupingExpressions == null) { - return; //shouldn't happen - the view should have been removed - } - } - if (setOp.getProperty(NodeConstants.Info.SET_OPERATION) != Operation.UNION) { - return; - } LinkedHashSet<AggregateSymbol> aggregates = collectAggregates(groupNode); Map<ElementSymbol, List<Set<Constant>>> partitionInfo = (Map<ElementSymbol, List<Set<Constant>>>)unionSourceParent.getProperty(Info.PARTITION_INFO); @@ -212,33 +189,10 @@ findUnionChildren(unionChildren, cardinalityDependent, setOp); SymbolMap parentMap = (SymbolMap)unionSourceParent.getProperty(NodeConstants.Info.SYMBOL_MAP); - List<ElementSymbol> virtualElements = parentMap.getKeys(); - GroupSymbol virtualGroup = unionSourceParent.getGroups().iterator().next(); - List<SingleElementSymbol> actualGroupingExpressions = groupingExpressions; - if (intermediateView != null) { - actualGroupingExpressions = new ArrayList<SingleElementSymbol>(groupingExpressions.size()); - SymbolMap viewMap = (SymbolMap)intermediateView.getProperty(NodeConstants.Info.SYMBOL_MAP); - for (SingleElementSymbol ses : groupingExpressions) { - Expression ex = viewMap.getMappedExpression((ElementSymbol)ses); - SingleElementSymbol newCol = null; - if (ex instanceof SingleElementSymbol) { - newCol = (SingleElementSymbol)ex; - } else { - newCol = new ExpressionSymbol("grouping", ex); //$NON-NLS-1$ - } - actualGroupingExpressions.add(newCol); - } - } - //partitioned union - if (partitionInfo != null && !Collections.disjoint(partitionInfo.keySet(), actualGroupingExpressions)) { - if (intermediateView != null) { - parentMap = pushGroupByView(plan, metadata, capFinder, unionSourceParent, - setOp, intermediateView, cardinalityDependent, - unionChildren, virtualElements, virtualGroup); - } - decomposeGroupBy(groupNode, unionSourceParent, groupingExpressions, aggregates, unionChildren, parentMap, metadata, capFinder, intermediateView != null); + if (partitionInfo != null && !Collections.disjoint(partitionInfo.keySet(), groupingExpressions)) { + decomposeGroupBy(groupNode, unionSourceParent, groupingExpressions, aggregates, unionChildren, parentMap, metadata, capFinder, context); return; } @@ -248,10 +202,15 @@ * serves as a hint to distribute a distinct to the union queries */ if (aggregates.isEmpty()) { - if (groupingExpressions != null && !groupingExpressions.isEmpty()) { + if (!groupingExpressions.isEmpty()) { setOp.setProperty(NodeConstants.Info.USE_ALL, Boolean.FALSE); } return; + } + for (AggregateSymbol agg : aggregates) { + if (!agg.canStage()) { + return; + } } //TODO: merge virtual, plan unions, raise null - change the partition information @@ -263,12 +222,12 @@ List<SingleElementSymbol> copy = new ArrayList<SingleElementSymbol>(aggregates); aggregates.clear(); Map<AggregateSymbol, Expression> aggMap = buildAggregateMap(copy, metadata, aggregates); - + boolean shouldPushdown = false; List<Boolean> pushdownList = new ArrayList<Boolean>(unionChildren.size()); for (PlanNode planNode : unionChildren) { - boolean pushdown = canPushGroupByToUnionChild(metadata, capFinder, actualGroupingExpressions, aggregates, planNode, record); + boolean pushdown = canPushGroupByToUnionChild(metadata, capFinder, groupingExpressions, aggregates, planNode, record); pushdownList.add(pushdown); shouldPushdown |= pushdown; } @@ -277,140 +236,238 @@ return; } - if (intermediateView != null) { - parentMap = pushGroupByView(plan, metadata, capFinder, unionSourceParent, - setOp, intermediateView, cardinalityDependent, - unionChildren, virtualElements, virtualGroup); - virtualElements = parentMap.getKeys(); - virtualGroup = unionSourceParent.getGroups().iterator().next(); - } + GroupSymbol group = unionSourceParent.getGroups().iterator().next().clone(); Iterator<Boolean> pushdownIterator = pushdownList.iterator(); + boolean first = true; for (PlanNode planNode : unionChildren) { - addView(plan, planNode, pushdownIterator.next(), new GroupSymbol("X"), groupingExpressions, aggregates, virtualElements, metadata, capFinder, null); //$NON-NLS-1$ + addUnionGroupBy(groupingExpressions, aggregates, parentMap, metadata, capFinder, context, group, first, planNode, !pushdownIterator.next()); + first = false; } - //update the parent plan with the staged aggregates and the new projected symbols - List<SingleElementSymbol> projectedViewSymbols = (List<SingleElementSymbol>)NodeEditor.findNodePreOrder(unionSourceParent, NodeConstants.Types.PROJECT).getProperty(NodeConstants.Info.PROJECT_COLS); + updateParentAggs(groupNode, context, aggMap, metadata); - //hack to introduce aggregate symbols to the parent view TODO: this should change the metadata properly. - SymbolMap newParentMap = modifyUnionSourceParent(unionSourceParent, virtualGroup, projectedViewSymbols, virtualElements); - Map<AggregateSymbol, ElementSymbol> projectedMap = new HashMap<AggregateSymbol, ElementSymbol>(); - Iterator<AggregateSymbol> aggIter = aggregates.iterator(); - for (ElementSymbol projectedViewSymbol : newParentMap.getKeys().subList(projectedViewSymbols.size() - aggregates.size(), projectedViewSymbols.size())) { - projectedMap.put(aggIter.next(), projectedViewSymbol); + List<SingleElementSymbol> symbols = (List<SingleElementSymbol>) NodeEditor.findNodePreOrder(unionSourceParent, NodeConstants.Types.PROJECT).getProperty(Info.PROJECT_COLS); + GroupSymbol modifiedGroup = group.clone(); + SymbolMap symbolMap = createSymbolMap(modifiedGroup, symbols, unionSourceParent, metadata); + unionSourceParent.setProperty(Info.SYMBOL_MAP, symbolMap); + + //correct the parent frame + Map<Expression, ElementSymbol> mapping = new HashMap<Expression, ElementSymbol>(); + Iterator<ElementSymbol> elemIter = symbolMap.getKeys().iterator(); + for (Expression expr : groupingExpressions) { + mapping.put(expr, elemIter.next()); } - for (Expression expr : aggMap.values()) { - ExpressionMappingVisitor.mapExpressions(expr, projectedMap); + for (AggregateSymbol agg : aggregates) { + mapping.put(agg, elemIter.next()); } - mapExpressions(groupNode.getParent(), aggMap, metadata); - } - - private SymbolMap pushGroupByView(PlanNode plan, - QueryMetadataInterface metadata, CapabilitiesFinder capFinder, - PlanNode unionSourceParent, PlanNode setOp, - PlanNode intermediateView, boolean cardinalityDependent, - LinkedList<PlanNode> unionChildren, - List<ElementSymbol> virtualElements, GroupSymbol virtualGroup) - throws TeiidComponentException, QueryPlannerException, - QueryResolverException { - //perform view pushing - /* - * TODO: this introduces yet another potentially unneeded view, but cannot be removed by the normal merge virtual logic - * due to an intervening access node - */ - PlanNode intermediateProject = intermediateView.getFirstChild(); - List<SingleElementSymbol> projectedViewSymbols = (List<SingleElementSymbol>)intermediateProject.getProperty(NodeConstants.Info.PROJECT_COLS); - for (PlanNode planNode : unionChildren) { - addView(plan, planNode, false, virtualGroup.clone(), null, Collections.EMPTY_SET, virtualElements, metadata, capFinder, LanguageObject.Util.deepClone(projectedViewSymbols, SingleElementSymbol.class)); + PlanNode node = unionSourceParent; + while (node != groupNode.getParent()) { + FrameUtil.convertNode(node, null, null, mapping, metadata, false); + node = node.getParent(); } - unionChildren.clear(); - findUnionChildren(unionChildren, cardinalityDependent, setOp); - virtualGroup = intermediateView.getGroups().iterator().next(); - unionSourceParent.getGroups().clear(); - unionSourceParent.addGroup(virtualGroup); - projectedViewSymbols = (List<SingleElementSymbol>)NodeEditor.findNodePreOrder(unionSourceParent, NodeConstants.Types.PROJECT).getProperty(NodeConstants.Info.PROJECT_COLS); - SymbolMap parentMap = modifyUnionSourceParent(unionSourceParent, virtualGroup, projectedViewSymbols, Collections.EMPTY_LIST); - //remove the old view - NodeEditor.removeChildNode(intermediateView, intermediateProject); - NodeEditor.removeChildNode(intermediateView.getParent(), intermediateView); - return parentMap; } - private SymbolMap modifyUnionSourceParent(PlanNode unionSourceParent, - GroupSymbol virtualGroup, - List<SingleElementSymbol> projectedViewSymbols, List<ElementSymbol> baseVirtualElements) { - List<ElementSymbol> updatedVirturalElement = new ArrayList<ElementSymbol>(baseVirtualElements); - for (int i = updatedVirturalElement.size(); i < projectedViewSymbols.size(); i++) { - SingleElementSymbol symbol = projectedViewSymbols.get(i); - String name = symbol.getShortName(); - String virtualElementName = virtualGroup.getCanonicalName() + ElementSymbol.SEPARATOR + name; - ElementSymbol virtualElement = new ElementSymbol(virtualElementName); - virtualElement.setGroupSymbol(virtualGroup); - virtualElement.setType(symbol.getType()); - virtualElement.setMetadataID(new TempMetadataID(virtualElementName, symbol.getType())); - updatedVirturalElement.add(virtualElement); + private void updateParentAggs(PlanNode groupNode, CommandContext context, + Map<AggregateSymbol, Expression> aggMap, QueryMetadataInterface metadata) + throws QueryMetadataException, TeiidComponentException, QueryPlannerException { + LinkedHashSet<AggregateSymbol> compositeAggs = new LinkedHashSet<AggregateSymbol>(); + boolean hasExpressionMapping = false; + for (Expression ex : aggMap.values()) { + if (ex instanceof AggregateSymbol) { + compositeAggs.add((AggregateSymbol) ex); + } else { + AggregateSymbolCollectorVisitor.getAggregates(ex, compositeAggs, null); + hasExpressionMapping = true; + } } - SymbolMap newParentMap = SymbolMap.createSymbolMap(updatedVirturalElement, projectedViewSymbols); - unionSourceParent.setProperty(NodeConstants.Info.SYMBOL_MAP, newParentMap); - return newParentMap; + if (!hasExpressionMapping) { + //if no new expressions are created we can just modify the existing aggregates + FrameUtil.correctSymbolMap(aggMap, groupNode); + } else { + //if new expressions are created we insert a view to handle the projection + groupNode.getGroups().clear(); + SymbolMap oldGroupingMap = (SymbolMap) groupNode.getProperty(Info.SYMBOL_MAP); + GroupSymbol oldGroup = oldGroupingMap.asMap().keySet().iterator().next().getGroupSymbol(); + SymbolMap groupingMap = RelationalPlanner.buildGroupingNode(compositeAggs, (List<? extends Expression>) groupNode.getProperty(Info.GROUP_COLS), groupNode, context, idGenerator); + ArrayList<SingleElementSymbol> projectCols = new ArrayList<SingleElementSymbol>(oldGroupingMap.asMap().size()); + SymbolMap correctedMap = new SymbolMap(); + Map<Expression, ElementSymbol> inverseMap = groupingMap.inserseMapping(); + for (Map.Entry<ElementSymbol, Expression> entry : oldGroupingMap.asMap().entrySet()) { + SingleElementSymbol ses = null; + if (entry.getValue() instanceof AggregateSymbol) { + Expression ex = aggMap.get(entry.getValue()); + if (ex instanceof AggregateSymbol) { + ses = inverseMap.get(ex); + } else { + ExpressionMappingVisitor.mapExpressions(ex, inverseMap); + ses = new ExpressionSymbol("expr", ex); //$NON-NLS-1$ + } + } else { + ses = inverseMap.get(entry.getValue()); + } + ses = (SingleElementSymbol) ses.clone(); + projectCols.add(new AliasSymbol(entry.getKey().getShortCanonicalName(), ses)); + correctedMap.addMapping(entry.getKey(), SymbolMap.getExpression(ses)); + } + PlanNode projectNode = groupNode.getParent(); + if (projectNode.getType() != NodeConstants.Types.PROJECT) { + projectNode = NodeFactory.getNewNode(NodeConstants.Types.PROJECT); + groupNode.addAsParent(projectNode); + projectNode.setProperty(Info.PROJECT_COLS, projectCols); + RuleDecomposeJoin.createSource(oldGroup, projectNode, correctedMap); + } else { + FrameUtil.convertFrame(projectNode, oldGroup, null, correctedMap.asMap(), metadata); + } + } } + /* if partitioned, then we don't need decomposition or the top level group by + * + * source + * set op + * project + * group [agg(x), {a, b}] + * source + * child 1 + * ... + * + */ private void decomposeGroupBy(PlanNode groupNode, PlanNode sourceNode, - List<SingleElementSymbol> groupingExpressions, + List<Expression> groupingExpressions, LinkedHashSet<AggregateSymbol> aggregates, LinkedList<PlanNode> unionChildren, SymbolMap parentMap, QueryMetadataInterface metadata, - CapabilitiesFinder capFinder, boolean hadIntermediateView) throws QueryPlannerException, QueryMetadataException, TeiidComponentException { + CapabilitiesFinder capFinder, CommandContext cc) throws QueryPlannerException, QueryMetadataException, TeiidComponentException, QueryResolverException { // remove the group node groupNode.getParent().replaceChild(groupNode, groupNode.getFirstChild()); + GroupSymbol group = sourceNode.getGroups().iterator().next().clone(); boolean first = true; - List<SingleElementSymbol> symbols = null; for (PlanNode planNode : unionChildren) { - PlanNode groupClone = NodeFactory.getNewNode(NodeConstants.Types.GROUP); - groupClone.setProperty(Info.GROUP_COLS, LanguageObject.Util.deepClone(groupingExpressions, SingleElementSymbol.class)); - groupClone.addGroups(groupNode.getGroups()); - - PlanNode view = RuleDecomposeJoin.createSource(group, planNode, parentMap); - - view.addAsParent(groupClone); - - PlanNode projectPlanNode = NodeFactory.getNewNode(NodeConstants.Types.PROJECT); - - Select allSymbols = new Select(groupingExpressions); - allSymbols.addSymbols(aggregates); - if (first) { - first = false; - QueryRewriter.makeSelectUnique(allSymbols, false); - symbols = allSymbols.getProjectedSymbols(); - } - projectPlanNode.setProperty(NodeConstants.Info.PROJECT_COLS, allSymbols.getSymbols()); - projectPlanNode.addGroups(view.getGroups()); - - groupClone.addAsParent(projectPlanNode); - - if (hadIntermediateView) { - //drill down to the possible access node - planNode = planNode.getFirstChild().getFirstChild(); - } - if (planNode.getType() == NodeConstants.Types.ACCESS) { - //TODO: temporarily remove the access node so that the inline view could be removed if possible - while (RuleRaiseAccess.raiseAccessNode(planNode, planNode, metadata, capFinder, true, null) != null) { - //continue to raise - } - } + addUnionGroupBy(groupingExpressions, aggregates, + parentMap, metadata, capFinder, cc, group, first, + planNode, false); + first = false; } + List<SingleElementSymbol> symbols = (List<SingleElementSymbol>) NodeEditor.findNodePreOrder(sourceNode, NodeConstants.Types.PROJECT).getProperty(Info.PROJECT_COLS); GroupSymbol modifiedGroup = group.clone(); SymbolMap symbolMap = createSymbolMap(modifiedGroup, symbols, sourceNode, metadata); sourceNode.setProperty(Info.SYMBOL_MAP, symbolMap); + + //map from the anon group to the updated inline view group + SymbolMap map = (SymbolMap)groupNode.getProperty(Info.SYMBOL_MAP); + Map<Expression, ElementSymbol> inverse = map.inserseMapping(); + SymbolMap newMapping = (SymbolMap) NodeEditor.findNodePreOrder(sourceNode, NodeConstants.Types.GROUP).getProperty(Info.SYMBOL_MAP); - FrameUtil.convertFrame(sourceNode, group, Collections.singleton(modifiedGroup), symbolMap.inserseMapping(), metadata); + GroupSymbol oldGroup = null; + Map<ElementSymbol, ElementSymbol> updatedMapping = new HashMap<ElementSymbol, ElementSymbol>(); + for (Map.Entry<ElementSymbol, Expression> entry : symbolMap.asMap().entrySet()) { + Expression ex = newMapping.getMappedExpression((ElementSymbol) entry.getValue()); + ElementSymbol orig = inverse.get(ex); + oldGroup = orig.getGroupSymbol(); + updatedMapping.put(orig, entry.getKey()); + } + FrameUtil.convertFrame(sourceNode, oldGroup, Collections.singleton(modifiedGroup), updatedMapping, metadata); } + private void addUnionGroupBy( + List<Expression> groupingExpressions, + LinkedHashSet<AggregateSymbol> aggregates, SymbolMap parentMap, + QueryMetadataInterface metadata, CapabilitiesFinder capFinder, + CommandContext cc, GroupSymbol group, boolean first, PlanNode planNode, boolean viewOnly) + throws QueryMetadataException, TeiidComponentException, + QueryPlannerException, QueryResolverException { + List<Expression> groupingColumns = LanguageObject.Util.deepClone(groupingExpressions, Expression.class); + + //branches other than the first need to have their projected column names updated + if (!first) { + PlanNode sortNode = NodeEditor.findNodePreOrder(planNode, NodeConstants.Types.SORT, NodeConstants.Types.SOURCE); + List<SingleElementSymbol> sortOrder = null; + OrderBy orderBy = null; + if (sortNode != null) { + orderBy = (OrderBy)sortNode.getProperty(Info.SORT_ORDER); + sortOrder = orderBy.getSortKeys(); + } + List<SingleElementSymbol> projectCols = FrameUtil.findTopCols(planNode); + List<ElementSymbol> virtualElements = parentMap.getKeys(); + for (int i = 0; i < virtualElements.size(); i++) { + ElementSymbol virtualElem = virtualElements.get(i); + SingleElementSymbol projectedSymbol = projectCols.get(i); + if (!projectedSymbol.getShortCanonicalName().equals(virtualElem.getShortCanonicalName())) { + if (sortOrder != null) { + int sortIndex = sortOrder.indexOf(projectedSymbol); + if (sortIndex > -1) { + updateSymbolName(sortOrder, sortIndex, virtualElem, sortOrder.get(sortIndex)); + orderBy.getOrderByItems().get(sortIndex).setSymbol(sortOrder.get(sortIndex)); + } + } + updateSymbolName(projectCols, i, virtualElem, projectedSymbol); + } + } + } + + PlanNode view = RuleDecomposeJoin.createSource(group, planNode, parentMap); + + PlanNode projectPlanNode = NodeFactory.getNewNode(NodeConstants.Types.PROJECT); + + Select allSymbols = new Select(); + for (Expression expr : groupingColumns) { + allSymbols.addSymbol(new ExpressionSymbol("expr", expr)); //$NON-NLS-1$ + } + if (viewOnly) { + for (AggregateSymbol agg : aggregates) { + agg = (AggregateSymbol)agg.clone(); + if (agg.getAggregateFunction() == Type.COUNT) { + if (agg.getExpression() == null) { + allSymbols.addSymbol(new ExpressionSymbol("stagedAgg", new Constant(1))); //$NON-NLS-1$ + } else { + SearchedCaseExpression count = new SearchedCaseExpression(Arrays.asList(new IsNullCriteria(agg.getExpression())), Arrays.asList(new Constant(Integer.valueOf(0)))); + count.setElseExpression(new Constant(Integer.valueOf(1))); + count.setType(DataTypeManager.DefaultDataClasses.INTEGER); + allSymbols.addSymbol(new ExpressionSymbol("stagedAgg", count)); //$NON-NLS-1$ + } + } else { //min, max, sum + Expression ex = agg.getExpression(); + ex = ResolverUtil.convertExpression(ex, DataTypeManager.getDataTypeName(agg.getType()), metadata); + allSymbols.addSymbol(new ExpressionSymbol("stagedAgg", ex)); //$NON-NLS-1$ + } + } + } else { + allSymbols.addSymbols(aggregates); + } + if (first) { + QueryRewriter.makeSelectUnique(allSymbols, false); + } + projectPlanNode.setProperty(NodeConstants.Info.PROJECT_COLS, allSymbols.getSymbols()); + projectPlanNode.addGroups(view.getGroups()); + + view.addAsParent(projectPlanNode); + + if (!viewOnly) { + addGroupBy(cc, view, groupingColumns, aggregates, metadata, projectPlanNode.getParent()); + } + + if (planNode.getType() == NodeConstants.Types.ACCESS) { + //TODO: temporarily remove the access node so that the inline view could be removed if possible + while (RuleRaiseAccess.raiseAccessNode(planNode, planNode, metadata, capFinder, true, null) != null) { + //continue to raise + } + } + } + + private void updateSymbolName(List<SingleElementSymbol> projectCols, int i, + ElementSymbol virtualElem, SingleElementSymbol projectedSymbol) { + if (projectedSymbol instanceof AliasSymbol) { + ((AliasSymbol)projectedSymbol).setShortName(virtualElem.getShortCanonicalName()); + } else { + projectCols.set(i, new AliasSymbol(virtualElem.getShortCanonicalName(), projectedSymbol)); + } + } + private boolean canPushGroupByToUnionChild(QueryMetadataInterface metadata, CapabilitiesFinder capFinder, - List<SingleElementSymbol> groupingExpressions, + List<Expression> groupingExpressions, LinkedHashSet<AggregateSymbol> aggregates, PlanNode planNode, AnalysisRecord record) throws QueryMetadataException, TeiidComponentException { if (planNode.getType() != NodeConstants.Types.ACCESS) { @@ -422,16 +479,16 @@ return false; } for (AggregateSymbol aggregate : aggregates) { - if (!CapabilitiesUtil.supportsAggregateFunction(modelId, aggregate, metadata, capFinder)) { - return false; - } + if(! CriteriaCapabilityValidatorVisitor.canPushLanguageObject(aggregate, modelId, metadata, capFinder, record)) { + return false; + } } - if ((groupingExpressions == null || groupingExpressions.isEmpty())) { + if (groupingExpressions.isEmpty()) { if (!CapabilitiesUtil.supports(Capability.QUERY_AGGREGATES_COUNT_STAR, modelId, metadata, capFinder)) { return false; } } else { - for (SingleElementSymbol ses : groupingExpressions) { + for (Expression ses : groupingExpressions) { if(! CriteriaCapabilityValidatorVisitor.canPushLanguageObject(ses, modelId, metadata, capFinder, record)) { return false; } @@ -466,109 +523,7 @@ return null; } - public void addView(PlanNode root, PlanNode unionSource, boolean pushdown, GroupSymbol group, List<SingleElementSymbol> groupingExpressions, - Set<AggregateSymbol> aggregates, List<ElementSymbol> virtualElements, - QueryMetadataInterface metadata, CapabilitiesFinder capFinder, List<SingleElementSymbol> actualProject) - throws TeiidComponentException, QueryPlannerException, QueryResolverException { - PlanNode accessNode = null; - if (pushdown) { - accessNode = NodeEditor.findNodePreOrder(unionSource, NodeConstants.Types.ACCESS); - } - //branches other than the first need to have their projected column names updated - PlanNode sortNode = NodeEditor.findNodePreOrder(unionSource, NodeConstants.Types.SORT, NodeConstants.Types.SOURCE); - List<SingleElementSymbol> sortOrder = null; - OrderBy orderBy = null; - if (sortNode != null) { - orderBy = (OrderBy)sortNode.getProperty(Info.SORT_ORDER); - sortOrder = orderBy.getSortKeys(); - } - List<SingleElementSymbol> projectCols = FrameUtil.findTopCols(unionSource); - for (int i = 0; i < virtualElements.size(); i++) { - ElementSymbol virtualElem = virtualElements.get(i); - SingleElementSymbol projectedSymbol = projectCols.get(i); - if (!projectedSymbol.getShortCanonicalName().equals(virtualElem.getShortCanonicalName())) { - if (sortOrder != null) { - int sortIndex = sortOrder.indexOf(projectedSymbol); - if (sortIndex > -1) { - updateSymbolName(sortOrder, sortIndex, virtualElem, sortOrder.get(sortIndex)); - orderBy.getOrderByItems().get(sortIndex).setSymbol(sortOrder.get(sortIndex)); - } - } - updateSymbolName(projectCols, i, virtualElem, projectedSymbol); - } - } - PlanNode intermediateView = createView(group, virtualElements, unionSource, metadata); - SymbolMap symbolMap = (SymbolMap)intermediateView.getProperty(Info.SYMBOL_MAP); - unionSource = intermediateView; - - Set<SingleElementSymbol> newGroupingExpressions = Collections.emptySet(); - if (groupingExpressions != null) { - newGroupingExpressions = new HashSet<SingleElementSymbol>(); - for (SingleElementSymbol singleElementSymbol : groupingExpressions) { - newGroupingExpressions.add(symbolMap.getKeys().get(virtualElements.indexOf(singleElementSymbol)).clone()); - } - } - - List<SingleElementSymbol> projectedViewSymbols = Util.deepClone(symbolMap.getKeys(), SingleElementSymbol.class); - - PlanNode parent = NodeEditor.findParent(unionSource, NodeConstants.Types.SOURCE); - SymbolMap parentMap = (SymbolMap) parent.getProperty(NodeConstants.Info.SYMBOL_MAP); - SymbolMap viewMapping = SymbolMap.createSymbolMap(parentMap.getKeys(), projectedViewSymbols); - for (AggregateSymbol agg : aggregates) { - agg = (AggregateSymbol)agg.clone(); - ExpressionMappingVisitor.mapExpressions(agg, viewMapping.asMap()); - if (pushdown) { - projectedViewSymbols.add(agg); - } else { - if (agg.getAggregateFunction() == Type.COUNT) { - if (agg.getExpression() == null) { - projectedViewSymbols.add(new ExpressionSymbol("stagedAgg", new Constant(1))); //$NON-NLS-1$ - } else { - SearchedCaseExpression count = new SearchedCaseExpression(Arrays.asList(new IsNullCriteria(agg.getExpression())), Arrays.asList(new Constant(Integer.valueOf(0)))); - count.setElseExpression(new Constant(Integer.valueOf(1))); - count.setType(DataTypeManager.DefaultDataClasses.INTEGER); - projectedViewSymbols.add(new ExpressionSymbol("stagedAgg", count)); //$NON-NLS-1$ - } - } else { //min, max, sum - Expression ex = agg.getExpression(); - ex = ResolverUtil.convertExpression(ex, DataTypeManager.getDataTypeName(agg.getType()), metadata); - projectedViewSymbols.add(new ExpressionSymbol("stagedAgg", ex)); //$NON-NLS-1$ - } - } - } - - if (pushdown) { - unionSource = addGroupBy(unionSource, newGroupingExpressions, new LinkedList<AggregateSymbol>()); - } - - PlanNode projectPlanNode = NodeFactory.getNewNode(NodeConstants.Types.PROJECT); - unionSource.addAsParent(projectPlanNode); - unionSource = projectPlanNode; - - //create proper names for the aggregate symbols - Select select = null; - if (actualProject == null) { - select = new Select(projectedViewSymbols); - } else { - select = new Select(actualProject); - } - QueryRewriter.makeSelectUnique(select, false); - projectedViewSymbols = select.getProjectedSymbols(); - projectPlanNode.setProperty(NodeConstants.Info.PROJECT_COLS, projectedViewSymbols); - projectPlanNode.addGroup(group); - if (pushdown) { - while (RuleRaiseAccess.raiseAccessNode(root, accessNode, metadata, capFinder, true, null) != null) { - //continue to raise - } - } - } - - static PlanNode createView(GroupSymbol group, List<? extends SingleElementSymbol> virtualElements, PlanNode child, QueryMetadataInterface metadata) throws TeiidComponentException { - SymbolMap symbolMap = createSymbolMap(group, virtualElements, child, metadata); - return RuleDecomposeJoin.createSource(group, child, symbolMap); - } - - private static SymbolMap createSymbolMap(GroupSymbol group, + static SymbolMap createSymbolMap(GroupSymbol group, List<? extends SingleElementSymbol> virtualElements, PlanNode child, QueryMetadataInterface metadata) throws TeiidComponentException, QueryMetadataException { @@ -585,15 +540,6 @@ return symbolMap; } - private void updateSymbolName(List<SingleElementSymbol> projectCols, int i, - ElementSymbol virtualElem, SingleElementSymbol projectedSymbol) { - if (projectedSymbol instanceof AliasSymbol) { - ((AliasSymbol)projectedSymbol).setShortName(virtualElem.getShortCanonicalName()); - } else { - projectCols.set(i, new AliasSymbol(virtualElem.getShortCanonicalName(), projectedSymbol)); - } - } - /** * Walk up the plan from the GROUP node. Should encounter only (optionally) a SELECT and can stop at the PROJECT node. Need to * collect any AggregateSymbols used in the select criteria or projected columns. @@ -605,23 +551,34 @@ static LinkedHashSet<AggregateSymbol> collectAggregates(PlanNode groupNode) { LinkedHashSet<AggregateSymbol> aggregates = new LinkedHashSet<AggregateSymbol>(); PlanNode currentNode = groupNode.getParent(); + SymbolMap symbolMap = (SymbolMap) groupNode.getProperty(NodeConstants.Info.SYMBOL_MAP); + while (currentNode != null) { if (currentNode.getType() == NodeConstants.Types.PROJECT) { List<SingleElementSymbol> projectedSymbols = (List<SingleElementSymbol>)currentNode.getProperty(NodeConstants.Info.PROJECT_COLS); for (SingleElementSymbol symbol : projectedSymbols) { - aggregates.addAll(AggregateSymbolCollectorVisitor.getAggregates(symbol, true)); + mapAggregates(ElementCollectorVisitor.getAggregates(symbol, true), symbolMap, aggregates); } break; } if (currentNode.getType() == NodeConstants.Types.SELECT) { Criteria crit = (Criteria)currentNode.getProperty(NodeConstants.Info.SELECT_CRITERIA); - aggregates.addAll(AggregateSymbolCollectorVisitor.getAggregates(crit, true)); + mapAggregates(ElementCollectorVisitor.getAggregates(crit, true), symbolMap, aggregates); } currentNode = currentNode.getParent(); } return aggregates; } + + static void mapAggregates(Collection<ElementSymbol> symbols, SymbolMap map, Collection<? super AggregateSymbol> aggs) { + for (ElementSymbol es : symbols) { + Expression ex = map.getMappedExpression(es); + if (ex instanceof AggregateSymbol) { + aggs.add((AggregateSymbol) ex); + } + } + } /** * Attempt to push the group node below one or more joins, manipulating the parent plan as necessary. This may involve @@ -631,32 +588,39 @@ * @since 4.2 */ private void pushGroupNode(PlanNode groupNode, - List<SingleElementSymbol> groupingExpressions, + List<Expression> groupingExpressions, Set<AggregateSymbol> allAggregates, QueryMetadataInterface metadata, - CapabilitiesFinder capFinder) throws TeiidComponentException, + CapabilitiesFinder capFinder, CommandContext cc) throws TeiidComponentException, QueryMetadataException, QueryPlannerException { Map<PlanNode, List<AggregateSymbol>> aggregateMap = createNodeMapping(groupNode, allAggregates, true); if (aggregateMap == null) { return; } - Map<PlanNode, List<SingleElementSymbol>> groupingMap = createNodeMapping(groupNode, groupingExpressions, false); + Map<PlanNode, List<Expression>> groupingMap = createNodeMapping(groupNode, groupingExpressions, false); Set<PlanNode> possibleTargetNodes = new LinkedHashSet<PlanNode>(aggregateMap.keySet()); possibleTargetNodes.addAll(groupingMap.keySet()); - + for (Map.Entry<PlanNode, List<AggregateSymbol>> entry : aggregateMap.entrySet()) { + if (AggregateSymbol.areAggregatesCardinalityDependent(entry.getValue())) { + //can't change the cardinality on the other side of the join - + //unless it's a 1-1 join, in which case this optimization isn't needed + //TODO: make a better choice if there are multiple targets + possibleTargetNodes.clear(); + possibleTargetNodes.add(entry.getKey()); + break; + } + } for (PlanNode planNode : possibleTargetNodes) { - Set<SingleElementSymbol> stagedGroupingSymbols = new LinkedHashSet<SingleElementSymbol>(); - List<AggregateSymbol> aggregates = aggregateMap.get(planNode); + Set<Expression> stagedGroupingSymbols = new LinkedHashSet<Expression>(); + Collection<AggregateSymbol> aggregates = aggregateMap.get(planNode); if (!canPush(groupNode, stagedGroupingSymbols, planNode)) { continue; } - if (groupingExpressions != null) { - filterJoinColumns(stagedGroupingSymbols, planNode.getGroups(), groupingExpressions); - } + filterJoinColumns(stagedGroupingSymbols, planNode.getGroups(), groupingExpressions); collectSymbolsFromOtherAggregates(allAggregates, aggregates, planNode, stagedGroupingSymbols); @@ -668,7 +632,7 @@ } if (aggregates != null) { - stageAggregates(groupNode, metadata, stagedGroupingSymbols, aggregates); + aggregates = stageAggregates(groupNode, metadata, stagedGroupingSymbols, aggregates, cc); } else { aggregates = new ArrayList<AggregateSymbol>(1); } @@ -678,12 +642,15 @@ } //TODO: if aggregates is empty, then could insert a dup remove node instead - PlanNode stageGroup = addGroupBy(planNode, stagedGroupingSymbols, aggregates); - + PlanNode stageGroup = addGroupBy(cc, planNode, new ArrayList<Expression>(stagedGroupingSymbols), aggregates, metadata, groupNode.getParent()); + //check for push down - if (stageGroup.getFirstChild().getType() == NodeConstants.Types.ACCESS - && RuleRaiseAccess.canRaiseOverGroupBy(stageGroup, stageGroup.getFirstChild(), aggregates, metadata, capFinder, null)) { - RuleRaiseAccess.performRaise(null, stageGroup.getFirstChild(), stageGroup); + PlanNode accessNode = stageGroup.getFirstChild(); + if (accessNode.getType() == NodeConstants.Types.ACCESS + && RuleRaiseAccess.canRaiseOverGroupBy(stageGroup, accessNode, aggregates, metadata, capFinder, null)) { + accessNode.getGroups().clear(); + accessNode.getGroups().addAll(stageGroup.getGroups()); + RuleRaiseAccess.performRaise(null, accessNode, stageGroup); if (stagedGroupingSymbols.isEmpty()) { RuleRaiseAccess.performRaise(null, stageGroup.getParent(), stageGroup.getParent().getParent()); } @@ -691,16 +658,14 @@ } } - private PlanNode addGroupBy(PlanNode planNode, - Collection<SingleElementSymbol> stagedGroupingSymbols, - Collection<AggregateSymbol> aggregates) { + private PlanNode addGroupBy(CommandContext cc, + PlanNode child, List<Expression> stagedGroupingSymbols, + Collection<AggregateSymbol> aggregates, QueryMetadataInterface metadata, PlanNode endNode) throws QueryMetadataException, + TeiidComponentException, QueryPlannerException { PlanNode stageGroup = NodeFactory.getNewNode(NodeConstants.Types.GROUP); - planNode.addAsParent(stageGroup); - - if (!stagedGroupingSymbols.isEmpty()) { - stageGroup.setProperty(NodeConstants.Info.GROUP_COLS, new ArrayList<SingleElementSymbol>(stagedGroupingSymbols)); - stageGroup.addGroups(GroupsUsedByElementsVisitor.getGroups(stagedGroupingSymbols)); - } else { + child.addAsParent(stageGroup); + aggregates = new LinkedHashSet<AggregateSymbol>(aggregates); + if (stagedGroupingSymbols.isEmpty()) { // if the source has no rows we need to insert a select node with criteria count(*)>0 PlanNode selectNode = NodeFactory.getNewNode(NodeConstants.Types.SELECT); AggregateSymbol count = new AggregateSymbol("stagedAgg", NonReserved.COUNT, false, null); //$NON-NLS-1$ @@ -710,48 +675,60 @@ selectNode.setProperty(NodeConstants.Info.IS_HAVING, Boolean.TRUE); stageGroup.addAsParent(selectNode); } + + Map<Expression, ElementSymbol> reverseMapping = RelationalPlanner.buildGroupingNode(aggregates, stagedGroupingSymbols, stageGroup, cc, idGenerator).inserseMapping(); + GroupSymbol newGroup = reverseMapping.values().iterator().next().getGroupSymbol(); + PlanNode node = stageGroup.getParent(); + while (node != endNode) { + if (node.getType() == NodeConstants.Types.JOIN) { + node.getGroups().removeAll(FrameUtil.findJoinSourceNode(stageGroup.getFirstChild()).getGroups()); + node.getGroups().add(newGroup); + } + FrameUtil.convertNode(node, null, null, reverseMapping, metadata, false); + if (node.getType() == NodeConstants.Types.JOIN) { + //reset the left/right/non-equi join criteria + RuleChooseJoinStrategy.chooseJoinStrategy(node, metadata); + } + node = node.getParent(); + } return stageGroup; } - static void stageAggregates(PlanNode groupNode, + Set<AggregateSymbol> stageAggregates(PlanNode groupNode, QueryMetadataInterface metadata, - Collection<SingleElementSymbol> stagedGroupingSymbols, - Collection<AggregateSymbol> aggregates) throws TeiidComponentException, QueryPlannerException { + Set<Expression> stagedGroupingSymbols, + Collection<AggregateSymbol> aggregates, CommandContext context) throws TeiidComponentException, QueryPlannerException { //remove any aggregates that are computed over a group by column - Set<Expression> expressions = new HashSet<Expression>(); - for (SingleElementSymbol expression : stagedGroupingSymbols) { - expressions.add(SymbolMap.getExpression(expression)); - } - for (final Iterator<AggregateSymbol> iterator = aggregates.iterator(); iterator.hasNext();) { final AggregateSymbol symbol = iterator.next(); Expression expr = symbol.getExpression(); if (expr == null) { continue; } - if (expressions.contains(expr)) { + if (stagedGroupingSymbols.contains(expr)) { iterator.remove(); } } - if (!aggregates.isEmpty()) { - // Fix any aggregate expressions so they correctly recombine the staged aggregates - try { - Set<AggregateSymbol> newAggs = new HashSet<AggregateSymbol>(); - Map<AggregateSymbol, Expression> aggMap = buildAggregateMap(aggregates, metadata, newAggs); - mapExpressions(groupNode.getParent(), aggMap, metadata); - aggregates.clear(); - aggregates.addAll(newAggs); - } catch (QueryResolverException err) { - throw new TeiidComponentException(err); - } - } + if (aggregates.isEmpty()) { + return Collections.emptySet(); + } + // Fix any aggregate expressions so they correctly recombine the staged aggregates + Set<AggregateSymbol> newAggs = new HashSet<AggregateSymbol>(); + Map<AggregateSymbol, Expression> aggMap; + try { + aggMap = buildAggregateMap(aggregates, metadata, newAggs); + } catch (QueryResolverException e) { + throw new QueryPlannerException(e, e.getMessage()); + } + updateParentAggs(groupNode, context, aggMap, metadata); + return newAggs; } private void collectSymbolsFromOtherAggregates(Collection<AggregateSymbol> allAggregates, Collection<AggregateSymbol> aggregates, PlanNode current, - Set<SingleElementSymbol> stagedGroupingSymbols) { + Set<Expression> stagedGroupingSymbols) { Set<AggregateSymbol> otherAggs = new HashSet<AggregateSymbol>(allAggregates); if (aggregates != null) { otherAggs.removeAll(aggregates); @@ -772,7 +749,7 @@ * Ensures that we are only pushing through inner equi joins or cross joins. Also collects the necessary staged grouping symbols */ private boolean canPush(PlanNode groupNode, - Set<SingleElementSymbol> stagedGroupingSymbols, + Set<Expression> stagedGroupingSymbols, PlanNode planNode) { PlanNode parentJoin = planNode.getParent(); @@ -786,12 +763,12 @@ } if (planNode == parentJoin.getFirstChild()) { - if (parentJoin.hasCollectionProperty(NodeConstants.Info.LEFT_EXPRESSIONS) && !filterJoinColumns(stagedGroupingSymbols, groups, (List<SingleElementSymbol>)parentJoin.getProperty(NodeConstants.Info.LEFT_EXPRESSIONS))) { - return false; + if (parentJoin.hasCollectionProperty(NodeConstants.Info.LEFT_EXPRESSIONS)) { + filterJoinColumns(stagedGroupingSymbols, groups, (List<SingleElementSymbol>)parentJoin.getProperty(NodeConstants.Info.LEFT_EXPRESSIONS)); } } else { - if (parentJoin.hasCollectionProperty(NodeConstants.Info.RIGHT_EXPRESSIONS) && !filterJoinColumns(stagedGroupingSymbols, groups, (List<SingleElementSymbol>)parentJoin.getProperty(NodeConstants.Info.RIGHT_EXPRESSIONS))) { - return false; + if (parentJoin.hasCollectionProperty(NodeConstants.Info.RIGHT_EXPRESSIONS)) { + filterJoinColumns(stagedGroupingSymbols, groups, (List<SingleElementSymbol>)parentJoin.getProperty(NodeConstants.Info.RIGHT_EXPRESSIONS)); } } @@ -801,21 +778,17 @@ return true; } - private boolean filterJoinColumns(Set<SingleElementSymbol> stagedGroupingSymbols, + private void filterJoinColumns(Set<Expression> stagedGroupingSymbols, Set<GroupSymbol> groups, - List<SingleElementSymbol> symbols) { - for (SingleElementSymbol singleElementSymbol : symbols) { - if (!(singleElementSymbol instanceof ElementSymbol)) { - return false; + List<? extends Expression> symbols) { + for (Expression ex : symbols) { + if (groups.containsAll(GroupsUsedByElementsVisitor.getGroups(ex))) { + stagedGroupingSymbols.add(SymbolMap.getExpression(ex)); } - if (groups.contains(((ElementSymbol)singleElementSymbol).getGroupSymbol())) { - stagedGroupingSymbols.add(singleElementSymbol); - } } - return true; } - private <T extends SingleElementSymbol> Map<PlanNode, List<T>> createNodeMapping(PlanNode groupNode, + private <T extends Expression> Map<PlanNode, List<T>> createNodeMapping(PlanNode groupNode, Collection<T> expressions, boolean aggs) { Map<PlanNode, List<T>> result = new LinkedHashMap<PlanNode, List<T>>(); if (expressions == null) { @@ -825,11 +798,14 @@ if (aggs && ((AggregateSymbol)aggregateSymbol).getExpression() == null) { return null; //count(*) is not yet handled. a general approach would be count(*) => count(r.col) * count(l.col), but the logic here assumes a simpler initial mapping } + if (aggs && !((AggregateSymbol)aggregateSymbol).canStage()) { + continue; + } Set<GroupSymbol> groups = GroupsUsedByElementsVisitor.getGroups(aggregateSymbol); if (groups.isEmpty()) { continue; } - PlanNode originatingNode = FrameUtil.findOriginatingNode(groupNode, groups); + PlanNode originatingNode = FrameUtil.findOriginatingNode(groupNode.getFirstChild(), groups); if (originatingNode == null) { if (aggs) { return null; //should never happen @@ -855,6 +831,10 @@ continue; } + if (originatingNode.getType() != NodeConstants.Types.ACCESS) { + continue; //don't perform intermediate grouping + } + if (aggs && ((AggregateSymbol)aggregateSymbol).isDistinct()) { //TODO: support distinct continue; @@ -873,7 +853,7 @@ private static Map<AggregateSymbol, Expression> buildAggregateMap(Collection<? extends SingleElementSymbol> aggregateExpressions, QueryMetadataInterface metadata, Set<AggregateSymbol> nestedAggregates) throws QueryResolverException, TeiidComponentException { - Map<AggregateSymbol, Expression> aggMap = new HashMap<AggregateSymbol, Expression>(); + Map<AggregateSymbol, Expression> aggMap = new LinkedHashMap<AggregateSymbol, Expression>(); for (SingleElementSymbol symbol : aggregateExpressions) { AggregateSymbol partitionAgg = (AggregateSymbol)symbol; @@ -883,7 +863,6 @@ if (aggFunction == Type.COUNT) { //COUNT(x) -> CONVERT(SUM(COUNT(x)), INTEGER) AggregateSymbol newAgg = new AggregateSymbol("stagedAgg", NonReserved.SUM, false, partitionAgg); //$NON-NLS-1$ - // Build conversion function to convert SUM (which returns LONG) back to INTEGER Function convertFunc = new Function(FunctionLibrary.CONVERT, new Expression[] {newAgg, new Constant(DataTypeManager.getDataTypeName(partitionAgg.getType()))}); ResolverVisitor.resolveLanguageObject(convertFunc, metadata); @@ -945,10 +924,7 @@ nestedAggregates.add(countAgg); nestedAggregates.add(sumAgg); nestedAggregates.add(sumSqAgg); - } else if (aggFunction == Type.TEXTAGG) { - continue; - } - else { + } else { //AGG(X) -> AGG(AGG(X)) newExpression = new AggregateSymbol("stagedAgg", aggFunction.name(), false, partitionAgg); //$NON-NLS-1$ nestedAggregates.add(partitionAgg); @@ -959,22 +935,6 @@ return aggMap; } - static void mapExpressions(PlanNode node, Map<? extends Expression, ? extends Expression> exprMap, QueryMetadataInterface metadata) - throws QueryPlannerException { - - while (node != null) { - FrameUtil.convertNode(node, null, null, exprMap, metadata, true); - - switch (node.getType()) { - case NodeConstants.Types.SOURCE: - case NodeConstants.Types.GROUP: - return; - } - - node = node.getParent(); - } - } - /** * @see java.lang.Object#toString() * @since 4.2 Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushSelectCriteria.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushSelectCriteria.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RulePushSelectCriteria.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -43,6 +43,7 @@ import org.teiid.query.optimizer.relational.plantree.NodeEditor; import org.teiid.query.optimizer.relational.plantree.NodeFactory; import org.teiid.query.optimizer.relational.plantree.PlanNode; +import org.teiid.query.optimizer.relational.plantree.NodeConstants.Info; import org.teiid.query.resolver.util.AccessPattern; import org.teiid.query.sql.lang.CompoundCriteria; import org.teiid.query.sql.lang.Criteria; @@ -52,7 +53,6 @@ import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.util.SymbolMap; -import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; import org.teiid.query.sql.visitor.ElementCollectorVisitor; import org.teiid.query.sql.visitor.ValueIteratorProviderCollectorVisitor; import org.teiid.query.util.CommandContext; @@ -113,7 +113,7 @@ boolean moved = false; - if((critNode.getGroups().isEmpty() && critNode.getSubqueryContainers().isEmpty()) || !atBoundary(critNode, sourceNode)) { + if(critNode.hasBooleanProperty(Info.IS_PUSHED) || (critNode.getGroups().isEmpty() && critNode.getSubqueryContainers().isEmpty()) || !atBoundary(critNode, sourceNode)) { deadNodes.add(critNode); continue; } @@ -132,6 +132,16 @@ break; } } + case NodeConstants.Types.GROUP: + { + if (!critNode.hasBooleanProperty(NodeConstants.Info.IS_HAVING)) { + SymbolMap symbolMap = (SymbolMap) sourceNode.getProperty(NodeConstants.Info.SYMBOL_MAP); + FrameUtil.convertNode(critNode, null, null, symbolMap.asMap(), metadata, true); + NodeEditor.removeChildNode(critNode.getParent(), critNode); + sourceNode.getFirstChild().addAsParent(critNode); + moved = true; + } + } } if (!moved) { @@ -333,8 +343,6 @@ satisfyAccessPatterns(critNode, currentNode); } else if (FrameUtil.isOrderedLimit(currentNode)) { return currentNode; - } else if (currentNode.getType() == NodeConstants.Types.GROUP && critNode.hasBooleanProperty(NodeConstants.Info.IS_HAVING)) { - return currentNode; } } @@ -598,7 +606,7 @@ return false; } - if (!AggregateSymbolCollectorVisitor.getAggregates(converted, false).isEmpty()) { + if (!ElementCollectorVisitor.getAggregates(converted, false).isEmpty()) { result = Boolean.TRUE; } } Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseAccess.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseAccess.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseAccess.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -163,6 +163,8 @@ { Set<AggregateSymbol> aggregates = RulePushAggregates.collectAggregates(parentNode); if (canRaiseOverGroupBy(parentNode, accessNode, aggregates, metadata, capFinder, record)) { + accessNode.getGroups().clear(); + accessNode.getGroups().addAll(parentNode.getGroups()); return performRaise(rootNode, accessNode, parentNode); } return null; @@ -302,14 +304,14 @@ if(modelID == null) { return false; } - List<SingleElementSymbol> groupCols = (List<SingleElementSymbol>)groupNode.getProperty(NodeConstants.Info.GROUP_COLS); + List<Expression> groupCols = (List<Expression>)groupNode.getProperty(NodeConstants.Info.GROUP_COLS); if(!CapabilitiesUtil.supportsAggregates(groupCols, modelID, metadata, capFinder)) { recordDebug("cannot push group by, since group by is not supported by source", groupNode, record); //$NON-NLS-1$ return false; } if (groupCols != null) { - for (SingleElementSymbol singleElementSymbol : groupCols) { - if (!canPushSymbol(singleElementSymbol, false, modelID, metadata, capFinder, record)) { + for (Expression expr : groupCols) { + if (!canPushSymbol(expr, false, modelID, metadata, capFinder, record)) { return false; } } @@ -467,7 +469,7 @@ * @throws QueryMetadataException * @since 4.1.2 */ - private static boolean canPushSymbol(SingleElementSymbol symbol, boolean inSelectClause, Object modelID, + private static boolean canPushSymbol(Expression symbol, boolean inSelectClause, Object modelID, QueryMetadataInterface metadata, CapabilitiesFinder capFinder, AnalysisRecord record) throws TeiidComponentException, QueryMetadataException { Modified: trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseNull.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseNull.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/optimizer/relational/rules/RuleRaiseNull.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -44,6 +44,8 @@ import org.teiid.query.sql.lang.OrderByItem; import org.teiid.query.sql.lang.SetQuery; import org.teiid.query.sql.symbol.AliasSymbol; +import org.teiid.query.sql.symbol.ElementSymbol; +import org.teiid.query.sql.symbol.Expression; import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; @@ -277,7 +279,7 @@ NodeEditor.removeChildNode(joinNode.getParent(), joinNode); for (GroupSymbol group : nullNode.getGroups()) { - Map nullSymbolMap = FrameUtil.buildSymbolMap(group, null, metadata); + Map<ElementSymbol, Expression> nullSymbolMap = FrameUtil.buildSymbolMap(group, null, metadata); FrameUtil.convertFrame(frameStart, group, null, nullSymbolMap, metadata); } Modified: trunk/engine/src/main/java/org/teiid/query/parser/ParseInfo.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/parser/ParseInfo.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/parser/ParseInfo.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -37,9 +37,6 @@ public int referenceCount = 0; - // This gets set according to the current clause - public boolean aggregatesAllowed = false; - // treat a double quoted variable as variable instead of string public boolean ansiQuotedIdentifiers=Boolean.valueOf(System.getProperty("org.teiid.ansiQuotedIdentifiers", Boolean.TRUE.toString())).booleanValue(); //$NON-NLS-1$ Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/DependentCriteriaProcessor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -51,6 +51,7 @@ import org.teiid.query.sql.lang.SetCriteria; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.Expression; +import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.util.ValueIterator; @@ -90,11 +91,11 @@ DependentValueSource originalVs = (DependentValueSource)dependentNode.getContext().getVariableContext().getGlobalValue(valueSource); if (!originalVs.isDistinct()) { if (sortUtility == null) { - List<Expression> sortSymbols = new ArrayList<Expression>(dependentSetStates.size()); + List<SingleElementSymbol> sortSymbols = new ArrayList<SingleElementSymbol>(dependentSetStates.size()); List<Boolean> sortDirection = new ArrayList<Boolean>(sortSymbols.size()); for (int i = 0; i < dependentSetStates.size(); i++) { sortDirection.add(Boolean.valueOf(OrderBy.ASC)); - sortSymbols.add(dependentSetStates.get(i).valueExpression); + sortSymbols.add((SingleElementSymbol)dependentSetStates.get(i).valueExpression); } this.sortUtility = new SortUtility(originalVs.getTupleBuffer().createIndexedTupleSource(), sortSymbols, sortDirection, Mode.DUP_REMOVE, dependentNode.getBufferManager(), dependentNode.getConnectionID(), originalVs.getTupleBuffer().getSchema()); } Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/GroupingNode.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -22,8 +22,7 @@ package org.teiid.query.processor.relational; -import static org.teiid.query.analysis.AnalysisRecord.PROP_GROUP_COLS; -import static org.teiid.query.analysis.AnalysisRecord.PROP_SORT_MODE; +import static org.teiid.query.analysis.AnalysisRecord.*; import java.util.ArrayList; import java.util.Arrays; @@ -62,18 +61,19 @@ import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; -import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.symbol.TextLine; import org.teiid.query.sql.symbol.AggregateSymbol.Type; +import org.teiid.query.sql.util.SymbolMap; import org.teiid.query.util.CommandContext; public class GroupingNode extends RelationalNode { // Grouping columns set by the planner - private List sortElements; - private List sortTypes; + private List<Expression> sortElements; + private List<Boolean> sortTypes; private boolean removeDuplicates; + private SymbolMap outputMapping; // Collection phase private int phase = COLLECTION; @@ -87,8 +87,10 @@ // Group phase private AggregateFunction[] functions; + private int[] conditions; private List lastRow; private List currentGroupTuple; + private Evaluator eval; private static final int COLLECTION = 1; private static final int SORT = 2; @@ -127,12 +129,16 @@ * @param groupingElements * @since 4.2 */ - public void setGroupingElements(List groupingElements) { + public void setGroupingElements(List<Expression> groupingElements) { this.sortElements = groupingElements; if(groupingElements != null) { sortTypes = Collections.nCopies(groupingElements.size(), Boolean.valueOf(OrderBy.ASC)); } } + + public void setOutputMapping(SymbolMap outputMapping) { + this.outputMapping = outputMapping; + } @Override public void initialize(CommandContext context, BufferManager bufferManager, @@ -157,13 +163,20 @@ // Construct aggregate function state accumulators functions = new AggregateFunction[getElements().size()]; + conditions = new int[getElements().size()]; for(int i=0; i<getElements().size(); i++) { - SingleElementSymbol symbol = (SingleElementSymbol)getElements().get(i); + Expression symbol = (Expression) getElements().get(i); + if (this.outputMapping != null) { + symbol = outputMapping.getMappedExpression((ElementSymbol)symbol); + } Class<?> outputType = symbol.getType(); Class<?> inputType = symbol.getType(); + conditions[i] = -1; if(symbol instanceof AggregateSymbol) { AggregateSymbol aggSymbol = (AggregateSymbol) symbol; - + if (aggSymbol.getCondition() != null) { + conditions[i] = collectExpression(aggSymbol.getCondition()); + } if(aggSymbol.getExpression() == null) { functions[i] = new Count(); } else { @@ -198,7 +211,6 @@ break; default: functions[i] = new StatsFunction(function); - } if(aggSymbol.isDistinct() && !function.equals(NonReserved.MIN) && !function.equals(NonReserved.MAX)) { @@ -281,8 +293,6 @@ return new BatchCollector.BatchProducerTupleSource(sourceNode) { - Evaluator eval = new Evaluator(elementMap, getDataManager(), getContext()); - @Override protected List updateTuple(List tuple) throws ExpressionEvaluationException, BlockedException, TeiidComponentException { int columns = collectedExpressions.size(); @@ -300,14 +310,20 @@ } private void collectionPhase() { + eval = new Evaluator(elementMap, getDataManager(), getContext()); if(this.sortElements == null) { // No need to sort this.groupTupleSource = getCollectionTupleSource(); this.phase = GROUP; } else { - this.sortUtility = new SortUtility(getCollectionTupleSource(), sortElements, + //create a temporary positional schema + List<ElementSymbol> schema = new ArrayList<ElementSymbol>(); + for (int i = 0; i < collectedExpressions.size(); i++) { + schema.add(new ElementSymbol(String.valueOf(i))); + } + this.sortUtility = new SortUtility(getCollectionTupleSource(), schema.subList(0, sortElements.size()), sortTypes, removeDuplicates?Mode.DUP_REMOVE_SORT:Mode.SORT, getBufferManager(), - getConnectionID(), collectedExpressions); + getConnectionID(), schema); this.phase = SORT; } } @@ -357,7 +373,7 @@ } if(lastRow != null || sortElements == null) { // Close last group - List row = new ArrayList(functions.length); + List<Object> row = new ArrayList<Object>(functions.length); for(int i=0; i<functions.length; i++) { row.add( functions[i].getResult() ); } @@ -397,10 +413,13 @@ return true; } - private void updateAggregates(List tuple) + private void updateAggregates(List<?> tuple) throws TeiidComponentException, TeiidProcessingException { for(int i=0; i<functions.length; i++) { + if (conditions[i] != -1 && !Boolean.TRUE.equals(tuple.get(conditions[i]))) { + continue; + } functions[i].addInput(tuple); } } @@ -415,6 +434,9 @@ protected void getNodeString(StringBuffer str) { super.getNodeString(str); str.append(sortElements); + if (outputMapping != null) { + str.append(outputMapping); + } } public Object clone(){ @@ -423,6 +445,7 @@ clonedNode.sortElements = sortElements; clonedNode.sortTypes = sortTypes; clonedNode.removeDuplicates = removeDuplicates; + clonedNode.outputMapping = outputMapping; return clonedNode; } Modified: trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/processor/relational/SortUtility.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -45,6 +45,7 @@ import org.teiid.query.sql.lang.OrderBy; import org.teiid.query.sql.lang.OrderByItem; import org.teiid.query.sql.symbol.Expression; +import org.teiid.query.sql.symbol.SingleElementSymbol; /** @@ -152,7 +153,7 @@ this.comparator.setNullOrdering(nullOrderings); } - public SortUtility(TupleSource ts, List expressions, List<Boolean> types, + public SortUtility(TupleSource ts, List<? extends SingleElementSymbol> expressions, List<Boolean> types, Mode mode, BufferManager bufferManager, String connectionID, List schema) { this(ts, new OrderBy(expressions, types).getOrderByItems(), mode, bufferManager, connectionID, schema); } Modified: trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverUtil.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverUtil.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverUtil.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -333,7 +333,7 @@ List<SingleElementSymbol> knownElements = command.getProjectedQuery().getSelect().getProjectedSymbols(); boolean isSimpleQuery = false; - List fromClauseGroups = Collections.emptyList(); + List<GroupSymbol> fromClauseGroups = Collections.emptyList(); if (command instanceof Query) { Query query = (Query)command; Modified: trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/resolver/util/ResolverVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -57,6 +57,7 @@ import org.teiid.query.sql.lang.SubqueryCompareCriteria; import org.teiid.query.sql.lang.SubquerySetCriteria; import org.teiid.query.sql.navigator.PostOrderNavigator; +import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.CaseExpression; import org.teiid.query.sql.symbol.Constant; import org.teiid.query.sql.symbol.DerivedColumn; @@ -423,6 +424,17 @@ handleException(e); } } + + @Override + public void visit(AggregateSymbol obj) { + if (obj.getCondition() != null) { + try { + obj.setCondition(ResolverUtil.convertExpression(obj.getCondition(), DataTypeManager.DefaultDataTypes.BOOLEAN, metadata)); + } catch (QueryResolverException e) { + handleException(e); + } + } + } public TeiidComponentException getComponentException() { return this.componentException; Modified: trunk/engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/rewriter/QueryRewriter.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -148,6 +148,7 @@ import org.teiid.query.sql.symbol.Reference; import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SearchedCaseExpression; +import org.teiid.query.sql.symbol.SelectSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.symbol.AggregateSymbol.Type; import org.teiid.query.sql.util.SymbolMap; @@ -759,83 +760,17 @@ * Converts a group by with expressions into a group by with only element symbols and an inline view * @param query * @return - * @throws QueryValidatorException + * @throws TeiidProcessingException + * @throws TeiidComponentException */ - private Query rewriteGroupBy(Query query) throws TeiidComponentException, TeiidProcessingException{ + private Query rewriteGroupBy(Query query) throws TeiidComponentException, TeiidProcessingException { if (query.getGroupBy() == null) { return query; } if (isDistinctWithGroupBy(query)) { query.getSelect().setDistinct(false); } - // we check for group by expressions here to create an ANSI SQL plan - boolean hasExpression = false; - for (final Iterator iterator = query.getGroupBy().getSymbols().iterator(); !hasExpression && iterator.hasNext();) { - hasExpression = iterator.next() instanceof ExpressionSymbol; - } - if (!hasExpression) { - return query; - } - Select select = query.getSelect(); - GroupBy groupBy = query.getGroupBy(); - query.setGroupBy(null); - Criteria having = query.getHaving(); - query.setHaving(null); - OrderBy orderBy = query.getOrderBy(); - query.setOrderBy(null); - Limit limit = query.getLimit(); - query.setLimit(null); - Into into = query.getInto(); - query.setInto(null); - Set<Expression> newSelectColumns = new HashSet<Expression>(); - for (final Iterator iterator = groupBy.getSymbols().iterator(); iterator.hasNext();) { - newSelectColumns.add(SymbolMap.getExpression((SingleElementSymbol)iterator.next())); - } - Set<AggregateSymbol> aggs = new HashSet<AggregateSymbol>(); - aggs.addAll(AggregateSymbolCollectorVisitor.getAggregates(select, true)); - if (having != null) { - aggs.addAll(AggregateSymbolCollectorVisitor.getAggregates(having, true)); - } - for (AggregateSymbol aggregateSymbol : aggs) { - if (aggregateSymbol.getExpression() != null) { - Expression expr = aggregateSymbol.getExpression(); - newSelectColumns.add(SymbolMap.getExpression(expr)); - } - } - Select innerSelect = new Select(); - int index = 0; - for (Expression expr : newSelectColumns) { - if (expr instanceof SingleElementSymbol) { - innerSelect.addSymbol((SingleElementSymbol)expr); - } else { - innerSelect.addSymbol(new ExpressionSymbol("EXPR" + index++ , expr)); //$NON-NLS-1$ - } - } - query.setSelect(innerSelect); - Query outerQuery = null; - try { - outerQuery = QueryRewriter.createInlineViewQuery(new GroupSymbol("X"), query, metadata, query.getSelect().getProjectedSymbols()); //$NON-NLS-1$ - } catch (TeiidException err) { - throw new TeiidRuntimeException(err); - } - Iterator<SingleElementSymbol> iter = outerQuery.getSelect().getProjectedSymbols().iterator(); - HashMap<Expression, SingleElementSymbol> expressionMap = new HashMap<Expression, SingleElementSymbol>(); - for (SingleElementSymbol symbol : query.getSelect().getProjectedSymbols()) { - expressionMap.put(SymbolMap.getExpression(symbol), iter.next()); - } - ExpressionMappingVisitor.mapExpressions(groupBy, expressionMap); - outerQuery.setGroupBy(groupBy); - ExpressionMappingVisitor.mapExpressions(having, expressionMap); - outerQuery.setHaving(having); - ExpressionMappingVisitor.mapExpressions(orderBy, expressionMap); - outerQuery.setOrderBy(orderBy); - outerQuery.setLimit(limit); - ExpressionMappingVisitor.mapExpressions(select, expressionMap); - outerQuery.setSelect(select); - outerQuery.setInto(into); - outerQuery.setOption(query.getOption()); - query = outerQuery; - rewriteExpressions(innerSelect); + rewriteExpressions(query.getGroupBy()); return query; } @@ -848,7 +783,7 @@ for (SingleElementSymbol selectExpr : query.getSelect().getProjectedSymbols()) { selectExpressions.add(SymbolMap.getExpression(selectExpr)); } - for (SingleElementSymbol groupByExpr : (List<SingleElementSymbol>)groupBy.getSymbols()) { + for (Expression groupByExpr : groupBy.getSymbols()) { if (!selectExpressions.contains(groupByExpr)) { return false; } @@ -2311,6 +2246,17 @@ expression.setAggregateFunction(Type.MAX); } } + if (expression.getExpression() != null && expression.getCondition() != null && !expression.respectsNulls()) { + Expression cond = expression.getCondition(); + Expression ex = expression.getExpression(); + if (!(cond instanceof Criteria)) { + cond = new ExpressionCriteria(cond); + } + SearchedCaseExpression sce = new SearchedCaseExpression(Arrays.asList(cond), Arrays.asList(ex)); + sce.setType(ex.getType()); + expression.setCondition(null); + expression.setExpression(sce); + } return expression; } @@ -2756,7 +2702,7 @@ select.setSymbols(select.getProjectedSymbols()); - List symbols = select.getSymbols(); + List<SelectSymbol> symbols = select.getSymbols(); HashSet<String> uniqueNames = new HashSet<String>(); Modified: trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/LanguageVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -97,6 +97,7 @@ import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SearchedCaseExpression; import org.teiid.query.sql.symbol.TextLine; +import org.teiid.query.sql.symbol.WindowFunction; import org.teiid.query.sql.symbol.XMLAttributes; import org.teiid.query.sql.symbol.XMLElement; import org.teiid.query.sql.symbol.XMLForest; @@ -223,4 +224,6 @@ public void visit(AlterView obj) {} public void visit(AlterProcedure obj) {} public void visit(AlterTrigger obj) {} + + public void visit(WindowFunction windowFunction) {} } Modified: trunk/engine/src/main/java/org/teiid/query/sql/lang/GroupBy.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/lang/GroupBy.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/lang/GroupBy.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -23,13 +23,10 @@ package org.teiid.query.sql.lang; import java.util.ArrayList; -import java.util.Collection; -import java.util.Iterator; import java.util.List; import org.teiid.core.util.EquivalenceUtil; import org.teiid.core.util.HashCodeUtil; -import org.teiid.query.QueryPlugin; import org.teiid.query.sql.LanguageObject; import org.teiid.query.sql.LanguageVisitor; import org.teiid.query.sql.symbol.Expression; @@ -46,7 +43,7 @@ public class GroupBy implements LanguageObject { /** The set of expressions for the data elements to be group. */ - private List symbols; // List<Expression> + private List<Expression> symbols; // ========================================================================= // C O N S T R U C T O R S @@ -56,15 +53,15 @@ * Constructs a default instance of this class. */ public GroupBy() { - symbols = new ArrayList(); + symbols = new ArrayList<Expression>(); } /** * Constructs an instance of this class from an ordered set of symbols. * @param symbols The ordered list of {@link org.teiid.query.sql.symbol.ElementSymbol}s */ - public GroupBy( List symbols ) { - this.symbols = new ArrayList( symbols ); + public GroupBy( List<? extends Expression> symbols ) { + this.symbols = new ArrayList<Expression>( symbols ); } // ========================================================================= @@ -83,7 +80,7 @@ * Returns an ordered list of the symbols in the GROUP BY * @return List of {@link org.teiid.query.sql.symbol.ElementSymbol}s */ - public List getSymbols() { + public List<Expression> getSymbols() { return symbols; } @@ -97,19 +94,6 @@ } } - /** - * Replaces the existing set of symbols with a new collection of symbols - * @param symbols Collection of {@link org.teiid.query.sql.symbol.ElementSymbol}s - * to replace current symbols with - */ - public void replaceSymbols( Collection symbols ) { - if(symbols == null) { - throw new IllegalArgumentException(QueryPlugin.Util.getString("ERR.015.010.0003")); //$NON-NLS-1$ - } - - this.symbols = new ArrayList(symbols); - } - public void acceptVisitor(LanguageVisitor visitor) { visitor.visit(this); } @@ -123,15 +107,7 @@ * @return Deep copy of object */ public Object clone() { - List thisSymbols = getSymbols(); - List copySymbols = new ArrayList(thisSymbols.size()); - Iterator iter = thisSymbols.iterator(); - while(iter.hasNext()) { - Expression es = (Expression) iter.next(); - copySymbols.add(es.clone()); - } - - return new GroupBy(copySymbols); + return new GroupBy(LanguageObject.Util.deepClone(this.symbols, Expression.class)); } /** Modified: trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/navigator/PreOrPostOrderNavigator.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -102,6 +102,7 @@ import org.teiid.query.sql.symbol.ScalarSubquery; import org.teiid.query.sql.symbol.SearchedCaseExpression; import org.teiid.query.sql.symbol.TextLine; +import org.teiid.query.sql.symbol.WindowFunction; import org.teiid.query.sql.symbol.XMLAttributes; import org.teiid.query.sql.symbol.XMLElement; import org.teiid.query.sql.symbol.XMLForest; @@ -140,12 +141,12 @@ visitVisitor(obj); } } - public void visit(AggregateSymbol obj) { preVisitVisitor(obj); visitNode(obj.getExpression()); visitNode(obj.getOrderBy()); + visitNode(obj.getCondition()); postVisitVisitor(obj); } public void visit(AliasSymbol obj) { @@ -700,6 +701,15 @@ postVisitVisitor(obj); } + @Override + public void visit(WindowFunction obj) { + preVisitVisitor(obj); + visitNode(obj.getFunction()); + visitNodes(obj.getPartition()); + visitNode(obj.getOrderBy()); + postVisitVisitor(obj); + } + public static void doVisit(LanguageObject object, LanguageVisitor visitor, boolean order) { doVisit(object, visitor, order, false); } Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/AggregateSymbol.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -34,14 +34,7 @@ /** - * An aggregate symbol represents an aggregate function in the SELECT or HAVING clauses. It - * extends ExpressionSymbol as they have many things in common. The aggregate symbol is - * typically something like <code>SUM(stock.quantity * 2)</code>. There are five supported - * aggregate functions: COUNT, SUM, AVG, MIN, and MAX. Aggregate functions contain an expression - - * this data is managed by the super class, ExpressionSymbol. Aggregate functions may also - * specify a DISTINCT flag to indicate that duplicates should be ignored. The DISTINCT flag - * may be set for all five aggregate functions but is ignored for the computation of MIN and MAX. - * One special use of an aggregate symbol is for the symbol <code>COUNT(*)</code>. The * expression + * An aggregate symbol represents an aggregate function. The * expression * is encoded by setting the expression to null. This may ONLY be used with the COUNT function. * * The type of an aggregate symbol depends on the function and the type of the underlying @@ -69,12 +62,16 @@ STDDEV_POP, STDDEV_SAMP, VAR_POP, - VAR_SAMP; + VAR_SAMP, + RANK, + DENSE_RANK, + ROW_NUMBER; } private Type aggregate; private boolean distinct; private OrderBy orderBy; + private Expression condition; private static final Class<Integer> COUNT_TYPE = DataTypeManager.DefaultDataClasses.INTEGER; private static final Map<Class<?>, Class<?>> SUM_TYPES; @@ -179,6 +176,8 @@ return DataTypeManager.DefaultDataClasses.DOUBLE; } else if (this.aggregate == Type.ARRAY_AGG) { return DataTypeManager.DefaultDataClasses.OBJECT; + } else if (this.aggregate == Type.RANK || this.aggregate == Type.ROW_NUMBER || this.aggregate == Type.DENSE_RANK){ + return DataTypeManager.DefaultDataClasses.INTEGER; } else { return this.getExpression().getType(); } @@ -222,6 +221,9 @@ if (orderBy != null) { copy.setOrderBy(orderBy.clone()); } + if (condition != null) { + copy.setCondition((Expression) condition.clone()); + } return copy; } @@ -246,6 +248,7 @@ return this.aggregate.equals(other.aggregate) && this.distinct == other.distinct && EquivalenceUtil.areEqual(this.getExpression(), other.getExpression()) + && EquivalenceUtil.areEqual(this.condition, other.condition) && EquivalenceUtil.areEqual(this.getOrderBy(), other.getOrderBy()); } @@ -261,10 +264,19 @@ case VAR_POP: case VAR_SAMP: case SUM: + case ARRAY_AGG: return true; } return false; } + + public Expression getCondition() { + return condition; + } + + public void setCondition(Expression condition) { + this.condition = condition; + } public static boolean areAggregatesCardinalityDependent(Collection<AggregateSymbol> aggs) { for (AggregateSymbol aggregateSymbol : aggs) { @@ -274,5 +286,21 @@ } return false; } + + public boolean respectsNulls() { + return this.aggregate == Type.ARRAY_AGG; + } + + public boolean canStage() { + switch (this.aggregate) { + case TEXTAGG: + return false; + case ARRAY_AGG: + return false; + case XMLAGG: + return orderBy == null; + } + return true; + } } Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/AliasSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/AliasSymbol.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/AliasSymbol.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -82,7 +82,7 @@ * Get the type of the symbol * @return Type of the symbol */ - public Class getType() { + public Class<?> getType() { return this.symbol.getType(); } Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/ElementSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/ElementSymbol.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/ElementSymbol.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -46,7 +46,8 @@ private GroupSymbol groupSymbol; private Object metadataID; private Class<?> type; - private boolean isExternalReference = false; + private boolean isExternalReference; + private boolean isAggregate; private DisplayMode displayMode = DisplayMode.OUTPUT_NAME; @@ -262,7 +263,16 @@ copy.setIsExternalReference(isExternalReference()); copy.outputName = this.outputName; copy.setDisplayMode(this.getDisplayMode()); + copy.isAggregate = isAggregate; return copy; } + public boolean isAggregate() { + return isAggregate; + } + + public void setAggregate(boolean isAggregate) { + this.isAggregate = isAggregate; + } + } Modified: trunk/engine/src/main/java/org/teiid/query/sql/symbol/ExpressionSymbol.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/ExpressionSymbol.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/ExpressionSymbol.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -34,7 +34,6 @@ */ public class ExpressionSymbol extends SingleElementSymbol { private Expression expression; - private boolean derivedExpression; /** * Constructor used for cloning @@ -101,19 +100,9 @@ clonedExpr = (Expression) getExpression().clone(); } ExpressionSymbol copy = new ExpressionSymbol(getName(), getCanonical(), clonedExpr); - copy.setDerivedExpression(this.derivedExpression); return copy; } - public boolean isDerivedExpression() { - return this.derivedExpression; - } - - public void setDerivedExpression(boolean derivedExpression) { - this.derivedExpression = derivedExpression; - } - - /** * @see org.teiid.query.sql.symbol.Symbol#hashCode() */ Added: trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java (rev 0) +++ trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -0,0 +1,114 @@ +/* + * JBoss, Home of Professional Open Source. + * See the COPYRIGHT.txt file distributed with this work for information + * regarding copyright ownership. Some portions may be licensed + * to Red Hat, Inc. under one or more contributor license agreements. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA + * 02110-1301 USA. + */ + + package org.teiid.query.sql.symbol; + +import java.util.List; + +import org.teiid.core.util.EquivalenceUtil; +import org.teiid.core.util.HashCodeUtil; +import org.teiid.query.sql.LanguageObject; +import org.teiid.query.sql.LanguageVisitor; +import org.teiid.query.sql.lang.OrderBy; + +public class WindowFunction implements Expression { + + private AggregateSymbol function; + private List<Expression> partition; + private OrderBy orderBy; + + public WindowFunction() { + + } + + public AggregateSymbol getFunction() { + return function; + } + + public void setFunction(AggregateSymbol expression) { + this.function = expression; + } + + public List<Expression> getPartition() { + return partition; + } + + public void setPartition(List<Expression> grouping) { + this.partition = grouping; + } + + public OrderBy getOrderBy() { + return orderBy; + } + + public void setOrderBy(OrderBy orderBy) { + this.orderBy = orderBy; + } + + @Override + public Class<?> getType() { + return function.getType(); + } + + @Override + public boolean isResolved() { + return function.isResolved(); + } + + @Override + public void acceptVisitor(LanguageVisitor visitor) { + visitor.visit(this); + } + + @Override + public int hashCode() { + return HashCodeUtil.hashCode(function.hashCode(), partition, orderBy); + } + + @Override + public boolean equals(Object obj) { + if (obj == this) { + return true; + } + if (!(obj instanceof WindowFunction)) { + return false; + } + WindowFunction other = (WindowFunction)obj; + return EquivalenceUtil.areEqual(this.function, other.function) && + EquivalenceUtil.areEqual(this.partition, other.partition) && + EquivalenceUtil.areEqual(this.orderBy, other.orderBy); + } + + @Override + public WindowFunction clone() { + WindowFunction clone = new WindowFunction(); + clone.setFunction((AggregateSymbol) this.function.clone()); + if (this.partition != null) { + clone.setPartition(LanguageObject.Util.deepClone(this.partition, Expression.class)); + } + if (this.orderBy != null) { + clone.setOrderBy(this.orderBy.clone()); + } + return clone; + } + +} Property changes on: trunk/engine/src/main/java/org/teiid/query/sql/symbol/WindowFunction.java ___________________________________________________________________ Added: svn:mime-type + text/plain Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/AggregateSymbolCollectorVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -32,7 +32,6 @@ import org.teiid.query.sql.navigator.PreOrPostOrderNavigator; import org.teiid.query.sql.symbol.AggregateSymbol; import org.teiid.query.sql.symbol.ElementSymbol; -import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; @@ -50,25 +49,12 @@ postVisitVisitor(obj); } - /** - * @see org.teiid.query.sql.navigator.PreOrPostOrderNavigator#visit(org.teiid.query.sql.symbol.ExpressionSymbol) - */ - @Override - public void visit(ExpressionSymbol obj) { - if (obj.isDerivedExpression()) { - preVisitVisitor(obj); - postVisitVisitor(obj); - } else { - super.visit(obj); - } - } - } - private Collection<AggregateSymbol> aggregates; - private Collection<SingleElementSymbol> groupingSymbols; + private Collection<? super AggregateSymbol> aggregates; + private Collection<? super SingleElementSymbol> groupingSymbols; - public AggregateSymbolCollectorVisitor(Collection<AggregateSymbol> aggregates, Collection<SingleElementSymbol> elements) { + public AggregateSymbolCollectorVisitor(Collection<? super AggregateSymbol> aggregates, Collection<? super SingleElementSymbol> elements) { this.aggregates = aggregates; this.groupingSymbols = elements; } @@ -79,23 +65,16 @@ } } - public void visit(ExpressionSymbol obj) { - if (this.groupingSymbols != null && obj.isDerivedExpression()) { - this.groupingSymbols.add(obj); - } - } - public void visit(ElementSymbol obj) { if (this.groupingSymbols != null) { this.groupingSymbols.add(obj); } } - public static final void getAggregates(LanguageObject obj, Collection<SingleElementSymbol> aggregates, Collection<SingleElementSymbol> elements) { - AggregateSymbolCollectorVisitor visitor = new AggregateSymbolCollectorVisitor(new ArrayList<AggregateSymbol>(), elements); + public static final void getAggregates(LanguageObject obj, Collection<? super AggregateSymbol> aggregates, Collection<SingleElementSymbol> elements) { + AggregateSymbolCollectorVisitor visitor = new AggregateSymbolCollectorVisitor(aggregates, elements); AggregateStopNavigator asn = new AggregateStopNavigator(visitor); obj.acceptVisitor(asn); - aggregates.addAll(visitor.aggregates); } public static final Collection<AggregateSymbol> getAggregates(LanguageObject obj, boolean removeDuplicates) { Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/ElementCollectorVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -49,7 +49,8 @@ */ public class ElementCollectorVisitor extends LanguageVisitor { - private Collection<ElementSymbol> elements; + private Collection<? super ElementSymbol> elements; + private boolean aggsOnly; /** * Construct a new visitor with the specified collection, which should @@ -57,7 +58,7 @@ * @param elements Collection to use for elements * @throws IllegalArgumentException If elements is null */ - public ElementCollectorVisitor(Collection<ElementSymbol> elements) { + public ElementCollectorVisitor(Collection<? super ElementSymbol> elements) { if(elements == null) { throw new IllegalArgumentException(QueryPlugin.Util.getString("ERR.015.010.0021")); //$NON-NLS-1$ } @@ -65,21 +66,14 @@ } /** - * Get the elements collected by the visitor. This should best be called - * after the visitor has been run on the language object tree. - * @return Collection of {@link org.teiid.query.sql.symbol.ElementSymbol} - */ - public Collection<ElementSymbol> getElements() { - return this.elements; - } - - /** * Visit a language object and collect symbols. This method should NOT be * called directly. * @param obj Language object */ public void visit(ElementSymbol obj) { - this.elements.add(obj); + if (!aggsOnly || obj.isAggregate()) { + this.elements.add(obj); + } } /** @@ -109,7 +103,7 @@ * @param obj Language object * @param elements Collection to collect elements in */ - public static final void getElements(LanguageObject obj, Collection<ElementSymbol> elements) { + public static final void getElements(LanguageObject obj, Collection<? super ElementSymbol> elements) { if(obj == null) { return; } @@ -150,6 +144,10 @@ * @return Collection of {@link org.teiid.query.sql.symbol.ElementSymbol} */ public static final Collection<ElementSymbol> getElements(LanguageObject obj, boolean removeDuplicates, boolean useDeepIteration) { + return getElements(obj, removeDuplicates, useDeepIteration, false); + } + + public static final Collection<ElementSymbol> getElements(LanguageObject obj, boolean removeDuplicates, boolean useDeepIteration, boolean aggsOnly) { if(obj == null) { return Collections.emptyList(); } @@ -159,16 +157,20 @@ } else { elements = new ArrayList<ElementSymbol>(); } - ElementCollectorVisitor visitor = null; + ElementCollectorVisitor visitor = new ElementCollectorVisitor(elements); + visitor.aggsOnly = aggsOnly; if (useDeepIteration){ - visitor = new ElementCollectorVisitor(elements); DeepPreOrderNavigator.doVisit(obj, visitor); } else { - visitor = new ElementCollectorVisitor(elements); PreOrderNavigator.doVisit(obj, visitor); } return elements; } + + public static final Collection<ElementSymbol> getAggregates(LanguageObject obj, boolean removeDuplicates) { + return getElements(obj, removeDuplicates, false, true); + } + } Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/ExpressionMappingVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/ExpressionMappingVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/ExpressionMappingVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -24,6 +24,7 @@ import java.util.ArrayList; import java.util.Collection; +import java.util.Collections; import java.util.HashSet; import java.util.Iterator; import java.util.List; @@ -52,6 +53,7 @@ import org.teiid.query.sql.lang.SubquerySetCriteria; import org.teiid.query.sql.lang.XMLTable; import org.teiid.query.sql.lang.XMLTable.XMLColumn; +import org.teiid.query.sql.navigator.PreOrPostOrderNavigator; import org.teiid.query.sql.navigator.PreOrderNavigator; import org.teiid.query.sql.proc.AssignmentStatement; import org.teiid.query.sql.symbol.AggregateSymbol; @@ -78,6 +80,7 @@ private Map symbolMap; private boolean clone = true; + private boolean elementSymbolsOnly; /** * Constructor for ExpressionMappingVisitor. @@ -251,7 +254,7 @@ public void visit(SearchedCaseExpression obj) { int whenCount = obj.getWhenCount(); - ArrayList thens = new ArrayList(whenCount); + ArrayList<Expression> thens = new ArrayList<Expression>(whenCount); for (int i = 0; i < whenCount; i++) { thens.add(replaceExpression(obj.getThenExpression(i))); } @@ -299,6 +302,9 @@ } public Expression replaceExpression(Expression element) { + if (elementSymbolsOnly && !(element instanceof ElementSymbol)) { + return element; + } Expression mapped = (Expression) this.symbolMap.get(element); if(mapped != null) { if (clone) { @@ -310,17 +316,20 @@ } public void visit(StoredProcedure obj) { - for (Iterator paramIter = obj.getInputParameters().iterator(); paramIter.hasNext();) { - SPParameter param = (SPParameter) paramIter.next(); + for (Iterator<SPParameter> paramIter = obj.getInputParameters().iterator(); paramIter.hasNext();) { + SPParameter param = paramIter.next(); Expression expr = param.getExpression(); param.setExpression(replaceExpression(expr)); } } public void visit(AggregateSymbol obj) { - if (obj.getExpression() != null) { //account for count(*) - TODO: clean this up + if (obj.getExpression() != null) { obj.setExpression(replaceExpression(obj.getExpression())); } + if (obj.getCondition() != null) { + obj.setCondition(replaceExpression(obj.getCondition())); + } } /** @@ -328,7 +337,11 @@ * @param obj Object to remap */ public void visit(GroupBy obj) { - replaceSymbols(obj.getSymbols(), false); + List<Expression> symbols = obj.getSymbols(); + for (int i = 0; i < symbols.size(); i++) { + Expression symbol = symbols.get(i); + symbols.set(i, replaceExpression(symbol)); + } } @Override @@ -371,21 +384,35 @@ * @param obj Language object * @param exprMap Expression map, Expression to Expression */ - public static void mapExpressions(LanguageObject obj, Map exprMap) { + public static void mapExpressions(LanguageObject obj, Map<? extends Expression, ? extends Expression> exprMap) { if(obj == null || exprMap == null || exprMap.isEmpty()) { return; } - final Set reverseSet = new HashSet(exprMap.values()); final ExpressionMappingVisitor visitor = new ExpressionMappingVisitor(exprMap); - PreOrderNavigator pon = new PreOrderNavigator(visitor) { - @Override - protected void visitNode(LanguageObject obj) { - if (!(obj instanceof Expression) || !reverseSet.contains(obj)) { - super.visitNode(obj); - } + visitor.elementSymbolsOnly = true; + for (Map.Entry<? extends Expression, ? extends Expression> entry : exprMap.entrySet()) { + if (!(entry.getKey() instanceof ElementSymbol)) { + visitor.elementSymbolsOnly = false; + break; } - }; - obj.acceptVisitor(pon); + } + boolean useReverseMapping = !Collections.disjoint(GroupsUsedByElementsVisitor.getGroups(exprMap.keySet()), + GroupsUsedByElementsVisitor.getGroups(exprMap.values())); + + if (useReverseMapping) { + final Set<Expression> reverseSet = new HashSet<Expression>(exprMap.values()); + PreOrderNavigator pon = new PreOrderNavigator(visitor) { + @Override + protected void visitNode(LanguageObject obj) { + if (!(obj instanceof Expression) || !reverseSet.contains(obj)) { + super.visitNode(obj); + } + } + }; + obj.acceptVisitor(pon); + } else { + PreOrPostOrderNavigator.doVisit(obj, visitor, PreOrPostOrderNavigator.PRE_ORDER, false); + } } protected void setVariableValues(Map variableValues) { Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/GroupsUsedByElementsVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/GroupsUsedByElementsVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/GroupsUsedByElementsVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -42,7 +42,9 @@ Collection<ElementSymbol> elements = ElementCollectorVisitor.getElements(obj, true); for (ElementSymbol elementSymbol : elements) { - groups.add(elementSymbol.getGroupSymbol()); + if (elementSymbol.getGroupSymbol() != null) { + groups.add(elementSymbol.getGroupSymbol()); + } } } Modified: trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/sql/visitor/SQLStringVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -126,6 +126,7 @@ import org.teiid.query.sql.symbol.SelectSymbol; import org.teiid.query.sql.symbol.SingleElementSymbol; import org.teiid.query.sql.symbol.TextLine; +import org.teiid.query.sql.symbol.WindowFunction; import org.teiid.query.sql.symbol.XMLAttributes; import org.teiid.query.sql.symbol.XMLElement; import org.teiid.query.sql.symbol.XMLForest; @@ -133,6 +134,7 @@ import org.teiid.query.sql.symbol.XMLParse; import org.teiid.query.sql.symbol.XMLQuery; import org.teiid.query.sql.symbol.XMLSerialize; +import org.teiid.query.sql.symbol.AggregateSymbol.Type; import org.teiid.query.sql.symbol.XMLNamespaces.NamespaceItem; import org.teiid.translator.SourceSystemFunctions; @@ -1123,7 +1125,9 @@ } if (obj.getExpression() == null) { - append(Tokens.ALL_COLS); + if (obj.getAggregateFunction() == Type.COUNT) { + append(Tokens.ALL_COLS); + } } else { visitNode(obj.getExpression()); } @@ -1133,6 +1137,16 @@ visitNode(obj.getOrderBy()); } append(")"); //$NON-NLS-1$ + + if (obj.getCondition() != null) { + append(SPACE); + append(FILTER); + append(Tokens.LPAREN); + append(WHERE); + append(SPACE); + append(obj.getCondition()); + append(Tokens.RPAREN); + } } public void visit( AliasSymbol obj ) { @@ -2034,6 +2048,31 @@ addTabs(0); append(alterView.getDefinition()); } + + @Override + public void visit(WindowFunction windowFunction) { + append(windowFunction.getFunction()); + append(SPACE); + append(OVER); + append(SPACE); + append(Tokens.LPAREN); + boolean needsSpace = false; + if (windowFunction.getPartition() != null) { + append(PARTITION); + append(SPACE); + append(BY); + append(SPACE); + registerNodes(windowFunction.getPartition(), 0); + needsSpace = true; + } + if (windowFunction.getOrderBy() != null) { + if (needsSpace) { + append(SPACE); + } + append(windowFunction.getOrderBy()); + } + append(Tokens.RPAREN); + } public static String escapeSinglePart( String part ) { if (isReservedWord(part)) { Modified: trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/util/CommandContext.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -434,13 +434,12 @@ } public Set<String> getGroups() { + if (globalState.groups == null) { + globalState.groups = new HashSet<String>(); + } return globalState.groups; } - public void setGroups(Set<String> groups) { - this.globalState.groups = groups; - } - public long getTimeSliceEnd() { return globalState.timeSliceEnd; } Modified: trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/validator/AggregateValidationVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -59,13 +59,9 @@ public void visit(AggregateSymbol obj) { Expression aggExp = obj.getExpression(); - // Check for any nested aggregates (which are not allowed) - if(aggExp != null) { - Collection<AggregateSymbol> nestedAggs = AggregateSymbolCollectorVisitor.getAggregates(aggExp, true); - if(nestedAggs.size() > 0) { - handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0039", nestedAggs), nestedAggs); //$NON-NLS-1$ - } - } + validateNoNestedAggs(aggExp); + validateNoNestedAggs(obj.getOrderBy()); + validateNoNestedAggs(obj.getCondition()); // Verify data type of aggregate expression Type aggregateFunction = obj.getAggregateFunction(); @@ -91,6 +87,16 @@ } validateBelow = false; } + + private void validateNoNestedAggs(LanguageObject aggExp) { + // Check for any nested aggregates (which are not allowed) + if(aggExp != null) { + Collection<AggregateSymbol> nestedAggs = AggregateSymbolCollectorVisitor.getAggregates(aggExp, true); + if(nestedAggs.size() > 0) { + handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0039", nestedAggs), nestedAggs); //$NON-NLS-1$ + } + } + } public void visit(ElementSymbol obj) { validateExpression(obj); Modified: trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java =================================================================== --- trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/java/org/teiid/query/validator/ValidationVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -114,7 +114,6 @@ import org.teiid.query.sql.symbol.DerivedColumn; import org.teiid.query.sql.symbol.ElementSymbol; import org.teiid.query.sql.symbol.Expression; -import org.teiid.query.sql.symbol.ExpressionSymbol; import org.teiid.query.sql.symbol.Function; import org.teiid.query.sql.symbol.GroupSymbol; import org.teiid.query.sql.symbol.QueryString; @@ -129,7 +128,6 @@ import org.teiid.query.sql.symbol.XMLParse; import org.teiid.query.sql.symbol.XMLQuery; import org.teiid.query.sql.symbol.AggregateSymbol.Type; -import org.teiid.query.sql.util.SymbolMap; import org.teiid.query.sql.visitor.AggregateSymbolCollectorVisitor; import org.teiid.query.sql.visitor.ElementCollectorVisitor; import org.teiid.query.sql.visitor.EvaluatableVisitor; @@ -208,18 +206,12 @@ public void visit(GroupBy obj) { // Get list of all group by IDs - List groupBySymbols = obj.getSymbols(); + List<Expression> groupBySymbols = obj.getSymbols(); validateSortable(groupBySymbols); - Iterator symbolIter = groupBySymbols.iterator(); - while(symbolIter.hasNext()) { - SingleElementSymbol symbol = (SingleElementSymbol)symbolIter.next(); - if(symbol instanceof ExpressionSymbol) { - ExpressionSymbol exprSymbol = (ExpressionSymbol) symbol; - Expression expr = exprSymbol.getExpression(); - if (!ValueIteratorProviderCollectorVisitor.getValueIteratorProviders(expr).isEmpty() || expr instanceof Constant || expr instanceof Reference) { - handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.groupby_subquery", expr), expr); //$NON-NLS-1$ - } - } + for (Expression expr : groupBySymbols) { + if (!ValueIteratorProviderCollectorVisitor.getValueIteratorProviders(expr).isEmpty() || expr instanceof Constant || expr instanceof Reference) { + handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.groupby_subquery", expr), expr); //$NON-NLS-1$ + } } } @@ -666,15 +658,13 @@ * and ORDER BY. * @param symbols List of SingleElementSymbol */ - protected void validateSortable(List symbols) { - Iterator iter = symbols.iterator(); - while(iter.hasNext()) { - SingleElementSymbol symbol = (SingleElementSymbol) iter.next(); - validateSortable(symbol); + protected void validateSortable(List<? extends Expression> symbols) { + for (Expression expression : symbols) { + validateSortable(expression); } } - private void validateSortable(SingleElementSymbol symbol) { + private void validateSortable(Expression symbol) { if (isNonComparable(symbol)) { handleValidationError(QueryPlugin.Util.getString("ERR.015.012.0026", symbol), symbol); //$NON-NLS-1$ } @@ -759,13 +749,16 @@ Select select = query.getSelect(); GroupBy groupBy = query.getGroupBy(); Criteria having = query.getHaving(); + validateNoAggsInClause(groupBy); + validateNoAggsInClause(query.getCriteria()); + validateNoAggsInClause(query.getFrom()); if(groupBy != null || having != null || !AggregateSymbolCollectorVisitor.getAggregates(select, false).isEmpty()) { Set<Expression> groupSymbols = null; if(groupBy != null) { groupSymbols = new HashSet<Expression>(); - for (final Iterator iterator = groupBy.getSymbols().iterator(); iterator.hasNext();) { - final SingleElementSymbol element = (SingleElementSymbol)iterator.next(); - groupSymbols.add(SymbolMap.getExpression(element)); + for (final Iterator<Expression> iterator = groupBy.getSymbols().iterator(); iterator.hasNext();) { + final Expression element = iterator.next(); + groupSymbols.add(element); } } @@ -783,10 +776,20 @@ // Move items to this report ValidatorReport report = visitor.getReport(); - Collection items = report.getItems(); + Collection<ValidatorFailure> items = report.getItems(); super.getReport().addItems(items); } } + + private void validateNoAggsInClause(LanguageObject clause) { + if (clause == null) { + return; + } + Collection<AggregateSymbol> aggs = AggregateSymbolCollectorVisitor.getAggregates(clause, false); + if (!aggs.isEmpty()) { + handleValidationError(QueryPlugin.Util.getString("SQLParser.Aggregate_only_top_level", aggs), aggs); + } + } protected void validateInsert(Insert obj) { Collection<ElementSymbol> vars = obj.getVariables(); @@ -1202,6 +1205,17 @@ @Override public void visit(AggregateSymbol obj) { + if (obj.getCondition() != null) { + Expression condition = obj.getCondition(); + if (!ValueIteratorProviderCollectorVisitor.getValueIteratorProviders(condition).isEmpty()) { + handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.filter_subquery", condition), condition); //$NON-NLS-1$ + } + for (ElementSymbol es : ElementCollectorVisitor.getElements(condition, false)) { + if (es.isExternalReference()) { + handleValidationError(QueryPlugin.Util.getString("ValidationVisitor.filter_subquery", es), es); //$NON-NLS-1$ + } + } + } if (obj.getAggregateFunction() != Type.TEXTAGG) { return; } Modified: trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj =================================================================== --- trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/javacc/org/teiid/query/parser/SQLParser.jj 2011-07-20 03:27:02 UTC (rev 3317) @@ -897,7 +897,7 @@ CriteriaSelector critSelector = new CriteriaSelector(); String element = null; - List elements = new ArrayList(); + List elements = new ArrayList(2); Token operator = null; } { @@ -1110,7 +1110,7 @@ [<WITH> { - critList = new ArrayList(); + critList = new ArrayList(2); } <LPAREN> element = id() @@ -1481,8 +1481,9 @@ }] ( - ( <VALUES> - values = rowValues(info) + ( <VALUES> <LPAREN> + values = expressionList(info) + <RPAREN> { // Store each row of values insert.setValues(values); @@ -1544,13 +1545,12 @@ * @return List of values, never null * @throws ParseException if parsing failed */ -List rowValues(ParseInfo info) : +ArrayList<Expression> expressionList(ParseInfo info) : { - List rowVals = new ArrayList(); + ArrayList<Expression> rowVals = new ArrayList<Expression>(4); Expression value = null; } { - <LPAREN> value = expression(info) { rowVals.add(value); @@ -1561,8 +1561,6 @@ rowVals.add(value); } )* - <RPAREN> - { return rowVals; } @@ -1822,7 +1820,6 @@ boolean isDistinct = false; // unless DISTINCT keyword in SELECT SelectSymbol symbol = null; Select select = new Select(); - info.aggregatesAllowed = true; } { <SELECT> @@ -1839,7 +1836,6 @@ ) ) { - info.aggregatesAllowed = false; select.setDistinct(isDistinct); return select; } @@ -1924,55 +1920,30 @@ } } -AggregateSymbol xmlAgg(ParseInfo info) : +AggregateSymbol orderedAgg(ParseInfo info) : { + Token t = null; Expression expression = null; OrderBy orderBy = null; + Expression condition = null; } { - <XMLAGG> <LPAREN> + (t=<XMLAGG>|t=<ARRAY_AGG>) + <LPAREN> expression = expression(info) - [ - orderBy = orderby(info) - ] + [ orderBy = orderby(info) ] <RPAREN> + condition = filterClause(info) { - if(! info.aggregatesAllowed) { - throw new ParseException(QueryPlugin.Util.getString("SQLParser.Aggregate_only_top_level")); //$NON-NLS-1$ - } - - String name = generateFunctionName(info, "XMLAGG"); - AggregateSymbol agg = new AggregateSymbol(name, "XMLAGG", false, expression); + String aggName = t.image.toUpperCase(); + String name = generateFunctionName(info, aggName); + AggregateSymbol agg = new AggregateSymbol(name, aggName, false, expression); agg.setOrderBy(orderBy); + agg.setCondition(condition); return agg; } } -AggregateSymbol arrayAgg(ParseInfo info) : -{ - Expression expression = null; - OrderBy orderBy = null; -} -{ - <ARRAY_AGG> <LPAREN> - expression = expression(info) - [ - orderBy = orderby(info) - ] - <RPAREN> - { - if(! info.aggregatesAllowed) { - throw new ParseException(QueryPlugin.Util.getString("SQLParser.Aggregate_only_top_level")); //$NON-NLS-1$ - } - - String name = generateFunctionName(info, "ARRAY_AGG"); - AggregateSymbol agg = new AggregateSymbol(name, "ARRAY_AGG", false, expression); - agg.setOrderBy(orderBy); - return agg; - } -} - - AggregateSymbol textAgg(ParseInfo info) : { DerivedColumn expression = null; @@ -1982,6 +1953,7 @@ List<DerivedColumn> expressions = new ArrayList<DerivedColumn>(); OrderBy orderBy = null; String encoding = null; + Expression condition = null; } { nonReserved("TEXTAGG") <LPAREN> @@ -2015,20 +1987,18 @@ orderBy = orderby(info) ] <RPAREN> + condition = filterClause(info) { - if(! info.aggregatesAllowed) { - throw new ParseException(QueryPlugin.Util.getString("SQLParser.Aggregate_only_top_level")); //$NON-NLS-1$ - } - TextLine tf = new TextLine(); tf.setDelimiter(delimiter); tf.setQuote(quote); tf.setIncludeHeader(header); tf.setExpressions(expressions); tf.setEncoding(encoding); - String name = generateFunctionName(info, "TEXTAGG"); + String name = generateFunctionName(info, "TEXTAGG"); AggregateSymbol agg = new AggregateSymbol(name, "TEXTAGG", false, tf); agg.setOrderBy(orderBy); + agg.setCondition(condition); return agg; } } @@ -2040,16 +2010,18 @@ AggregateSymbol agg = null; boolean isDistinct = false; Expression expression = null; + Expression condition = null; } { ( // COUNT(*) - LOOKAHEAD(3) ( + (LOOKAHEAD(3) ( func = nonReserved("COUNT") <LPAREN> starToken = <STAR> <RPAREN> ) | - + LOOKAHEAD(<ID>, {matchesAny(getToken(1).image, "rank", "dense_rank", "row_number") != null}) (func = nonReserved("row_number", "rank", "dense_rank", "percent_rank", "cume_dist") + <LPAREN> <RPAREN>) | // Remaining aggregates ( (func = nonReserved("COUNT", "SUM", "AVG", "MIN", "MAX", "EVERY", "STDDEV_POP", "STDDEV_SAMP", "VAR_SAMP", "VAR_POP") | @@ -2061,12 +2033,10 @@ [ <DISTINCT> {isDistinct=true;} | <ALL>] expression = expression(info) <RPAREN> - ) + )) + condition = filterClause(info) ) { - if(! info.aggregatesAllowed) { - throw new ParseException(QueryPlugin.Util.getString("SQLParser.Aggregate_only_top_level")); //$NON-NLS-1$ - } func = func.toUpperCase(); String name = generateFunctionName(info, func); if(starToken == null) { @@ -2076,10 +2046,22 @@ // COUNT(*) agg = new AggregateSymbol(name, func, false, null); } + agg.setCondition(condition); return agg; } } +Expression filterClause(ParseInfo info) : +{ + Expression condition = null; +} +{ + [ <FILTER> <LPAREN> <WHERE> condition = booleanPrimary(info) <RPAREN> ] + { + return condition; + } +} + /** * Parse a FROM. The from must handle groups, aliased groups or * joined groups. This also handles JDBC escape processinf syntax for outer joins. @@ -2631,7 +2613,7 @@ */ Criteria compoundCritOr(ParseInfo info) : { - ArrayList logicList = new ArrayList(); + ArrayList logicList = new ArrayList(2); Criteria logicPart = null; } { @@ -2654,7 +2636,7 @@ */ Criteria compoundCritAnd(ParseInfo info) : { - ArrayList logicList = new ArrayList(); + ArrayList logicList = new ArrayList(2); Criteria logicPart = null; } { @@ -2677,14 +2659,20 @@ */ Criteria notCrit(ParseInfo info) : { - Criteria crit = null; + Expression ex = null; boolean isNot = false; } { [<NOT> {isNot=true;}] - crit=booleanPrimary(info) + ex=booleanPrimary(info) { + Criteria crit = null; + if (ex instanceof Criteria) { + crit = (Criteria)ex; + } else { + crit = new ExpressionCriteria(ex); + } if(isNot) { return new NotCriteria(crit); } else { @@ -2698,38 +2686,30 @@ * @return criteria * @throws ParseException if parsing failed */ -Criteria booleanPrimary(ParseInfo info) : +Expression booleanPrimary(ParseInfo info) : { Expression ex = null; - Criteria crit = null; } { ( - LOOKAHEAD(2) crit = translateCriteria(info) + LOOKAHEAD(2) ex = translateCriteria(info) | (ex = commonValueExpression(info) - { - if (ex instanceof Criteria) { - crit = (Criteria)ex; - } else { - crit = new ExpressionCriteria(ex); - } - } [( - LOOKAHEAD(2) crit=betweenCrit(info, ex) | - LOOKAHEAD(2) crit=matchCrit(info, ex) | - crit=setCrit(info, ex) | - crit=isNullCrit(info, ex) | - LOOKAHEAD(operator() (<ANY>|<SOME>|<ALL>) subquery(info)) crit=subqueryCompareCriteria(info, ex) | - crit=compareCrit(info, ex) + LOOKAHEAD(2) ex=betweenCrit(info, ex) | + LOOKAHEAD(2) ex=matchCrit(info, ex) | + ex=setCrit(info, ex) | + ex=isNullCrit(info, ex) | + LOOKAHEAD(operator() (<ANY>|<SOME>|<ALL>) subquery(info)) ex=subqueryCompareCriteria(info, ex) | + ex=compareCrit(info, ex) )] ) | - crit=existsCriteria(info) | - crit = hasCriteria() + ex=existsCriteria(info) | + ex = hasCriteria() ) { - return crit; + return ex; } } @@ -3018,53 +2998,17 @@ */ GroupBy groupBy(ParseInfo info) : { - GroupBy groupBy = new GroupBy(); - SingleElementSymbol symbol = null; + List<Expression> expressions = null; } { <GROUP> <BY> - ( symbol = groupByItem(info) - { - groupBy.addSymbol(symbol); - } - - (<COMMA> symbol = groupByItem(info) - { - groupBy.addSymbol(symbol); - } - )* - ) + expressions = expressionList(info) { - return groupBy; + return new GroupBy(expressions); } } /** - * Parse a GROUP BY list item. - * @return Parsed group by item - * @throws ParseException if parsing failed - */ -SingleElementSymbol groupByItem(ParseInfo info) : -{ - Expression expr = null; - SingleElementSymbol symbol = null; -} -{ - expr = expression(info) - { - if(expr instanceof ElementSymbol) { - symbol = (ElementSymbol) expr; - } else { - String exprName = generateFunctionName(info, null); - symbol = new ExpressionSymbol(exprName, expr); - } - } - { - return symbol; - } -} - -/** * Parse a HAVING clause. * @return Parsed having * @throws ParseException if parsing failed @@ -3072,14 +3016,11 @@ Criteria having(ParseInfo info) : { Criteria criteria = null; - info.aggregatesAllowed = true; } { <HAVING> criteria = criteria(info) - { - info.aggregatesAllowed = false; return criteria; } } @@ -3438,19 +3379,20 @@ <RBRACE> ) | - LOOKAHEAD(<ID> <LPAREN> <FOR>) (expression=textAgg(info)) + LOOKAHEAD(<ID> <LPAREN>, {matchesAny(getToken(1).image, "textagg") != null}) + (expression=textAgg(info) [expression = windowSpecification(expression, info)]) | // Aggregate function - LOOKAHEAD(<ID>, {matchesAny(getToken(1).image, "count", "min", "max", "sum", "avg", "every", "STDDEV_POP", "STDDEV_SAMP", "VAR_SAMP", "VAR_POP") != null}) (expression=aggregateSymbol(info)) + LOOKAHEAD(<ID> <LPAREN>, {matchesAny(getToken(1).image, "count", "min", "max", "sum", "avg", "every", "STDDEV_POP", "STDDEV_SAMP", "VAR_SAMP", "VAR_POP") != null}) + (expression=aggregateSymbol(info) [expression = windowSpecification(expression, info)]) | - LOOKAHEAD(<ANY>) (expression=aggregateSymbol(info)) + LOOKAHEAD(<ANY>|<SOME>) (expression=aggregateSymbol(info) [expression = windowSpecification(expression, info)]) | - LOOKAHEAD(<SOME>) (expression=aggregateSymbol(info)) + expression=orderedAgg(info) [expression = windowSpecification(expression, info)] | - (expression=xmlAgg(info)) + LOOKAHEAD(<ID> <LPAREN>, {matchesAny(getToken(1).image, "rank", "dense_rank", "row_number") != null}) + (expression=aggregateSymbol(info) expression = windowSpecification(expression, info)) | - (expression=arrayAgg(info)) - | // Function LOOKAHEAD(2) (expression=function(info)) | @@ -3501,6 +3443,26 @@ } } +Expression windowSpecification(Expression agg, ParseInfo info) : +{ + List<Expression> partitionList = null; + OrderBy orderBy = null; +} +{ + <OVER> + <LPAREN> + [<PARTITION> <BY> partitionList = expressionList(info)] + [orderBy = orderby(info)] + <RPAREN> + { + WindowFunction result = new WindowFunction(); + result.setFunction((AggregateSymbol)agg); + result.setPartition(partitionList); + result.setOrderBy(orderBy); + return result; + } +} + /** * Parse a non-searched CASE expression. * @return CaseExpression @@ -3509,8 +3471,8 @@ CaseExpression caseExpression(ParseInfo info) : { Expression expression = null, whenExpression = null, thenExpression = null, elseExpression = null; - ArrayList whenExpressions = new ArrayList(); - ArrayList thenExpressions = new ArrayList(); + ArrayList whenExpressions = new ArrayList(2); + ArrayList thenExpressions = new ArrayList(2); } { <CASE> @@ -3543,8 +3505,8 @@ { Expression thenExpression = null, elseExpression = null; Criteria whenCriteria = null; - ArrayList whenCriteriaList = new ArrayList(); - ArrayList thenExpressions = new ArrayList(); + ArrayList whenCriteriaList = new ArrayList(2); + ArrayList thenExpressions = new ArrayList(2); } { <CASE> @@ -3578,7 +3540,7 @@ String funcName = null; Expression expression = null; - ArrayList args = new ArrayList(); + ArrayList args = new ArrayList(2); Token funcToken = null; } { @@ -3665,62 +3627,16 @@ | funcToken = <YEAR> | funcToken = <MONTH> | funcToken = <HOUR> | funcToken = <MINUTE> | funcToken = <SECOND> | funcToken = <XMLCONCAT> | funcToken = <XMLCOMMENT>) - <LPAREN> - [ - expression = expression(info) - { - args.add(expression); - expression = null; - } - - (<COMMA> expression=expression(info) - { - args.add(expression); - expression = null; - } - )* - ] + <LPAREN> + [args = expressionList(info)] <RPAREN> ) | - LOOKAHEAD(<INSERT> <LPAREN>) ( - (funcToken = <INSERT>) + LOOKAHEAD(<TRANSLATE>|<INSERT> <LPAREN>) ( + (funcToken = <TRANSLATE> | funcToken = <INSERT>) <LPAREN> - [ - expression = expression(info) - { - args.add(expression); - expression = null; - } - - (<COMMA> expression=expression(info) - { - args.add(expression); - expression = null; - } - )* - ] + [args = expressionList(info)] <RPAREN> - ) - | - LOOKAHEAD(<TRANSLATE> <LPAREN>) ( - (funcToken = <TRANSLATE>) - <LPAREN> - [ - expression = expression(info) - { - args.add(expression); - expression = null; - } - - (<COMMA> expression=expression(info) - { - args.add(expression); - expression = null; - } - )* - ] - <RPAREN> ) | expression = xmlParse(info) { @@ -3765,20 +3681,7 @@ | ( funcName = id() <LPAREN> - [ - expression = expression(info) - { - args.add(expression); - expression = null; - } - - (<COMMA> expression=expression(info) - { - args.add(expression); - expression = null; - } - )* - ] + [ args = expressionList(info) ] <RPAREN> )) { @@ -3830,7 +3733,7 @@ { Expression path = null; DerivedColumn arg = null; - ArrayList<DerivedColumn> args = new ArrayList<DerivedColumn>(); + ArrayList<DerivedColumn> args = new ArrayList<DerivedColumn>(2); } { nonReserved("QUERYSTRING") <LPAREN> @@ -3850,7 +3753,7 @@ XMLElement xmlElement(ParseInfo info) : { String name = null; - ArrayList content = new ArrayList(); + ArrayList content = new ArrayList(2); XMLNamespaces xmlNamespaces = null; XMLAttributes xmlAttributes = null; Expression expression = null; @@ -3886,7 +3789,7 @@ XMLAttributes xmlAttributes(ParseInfo info) : { DerivedColumn expression = null; - ArrayList<DerivedColumn> args = new ArrayList<DerivedColumn>(); + ArrayList<DerivedColumn> args = new ArrayList<DerivedColumn>(2); } { <XMLATTRIBUTES> <LPAREN> @@ -3911,7 +3814,7 @@ { DerivedColumn expression = null; XMLNamespaces xmlNamespaces = null; - ArrayList<DerivedColumn> args = new ArrayList<DerivedColumn>(); + ArrayList<DerivedColumn> args = new ArrayList<DerivedColumn>(2); } { <XMLFOREST> <LPAREN> @@ -3939,7 +3842,7 @@ XMLNamespaces xmlNamespaces(ParseInfo info) : { - ArrayList namespaces = new ArrayList(); + ArrayList namespaces = new ArrayList(2); XMLNamespaces.NamespaceItem item = null; } { Modified: trunk/engine/src/main/resources/org/teiid/query/i18n.properties =================================================================== --- trunk/engine/src/main/resources/org/teiid/query/i18n.properties 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/main/resources/org/teiid/query/i18n.properties 2011-07-20 03:27:02 UTC (rev 3317) @@ -259,7 +259,7 @@ SQLParser.Unknown_join_type=Unknown join type: {0} -SQLParser.Aggregate_only_top_level=Aggregate expressions are allowed only as top level functions in the SELECT and HAVING clauses. +SQLParser.Aggregate_only_top_level=Aggregate functions are not allowed in the FROM/WHERE/GROUP BY clauses: {0} SQLParser.Unknown_agg_func=Unknown aggregate function: {0} SQLParser.Invalid_func=Invalid function name: [{0}] SQLParser.Integer_parse=Unable to parse integer literal: {0} @@ -677,7 +677,8 @@ SimpleQueryResolver.procedure_cache_used=Procedure caching will be used for {0}. SimpleQueryResolver.procedure_cache_not_usable=Procedure caching will not be used for {0} since the result set cache is disabled or the results/parameters cannot be cached. SimpleQueryResolver.procedure_cache_not_used=Procedure caching will not be used for {0} due to the use of OPTION NOCACHE. -ValidationVisitor.groupby_subquery=Expressions used in a GROUP BY cannot be constant and must not contain subqueries: "{0}". +ValidationVisitor.groupby_subquery=Expressions used in a GROUP BY clause cannot be constant and must not contain subqueries: "{0}". +ValidationVisitor.filter_subquery=Expressions used in a FILTER clause must not contain subqueries nor outer references: "{0}". ValidationVisitor.Procedure_has_group_self_reference=Procedure cannot have a Group reference to itself. ExpressionEvaluator.Expected_props_for_payload_function=Unable to evaluate {0}: expected Properties for command payload but got object of type {1} ValidationVisitor.The_rowlimit_function_cannot_be_used_in_a_non-XML_command=The ''rowlimit'' and ''rowlimitexception'' functions cannot be used in a non-XML command Modified: trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java =================================================================== --- trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/dqp/internal/process/multisource/TestMultiSourcePlanToProcessConverter.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -35,7 +35,6 @@ import org.teiid.adminapi.impl.VDBMetaData; import org.teiid.common.buffer.TupleSource; import org.teiid.core.id.IDGenerator; -import org.teiid.core.id.IntegerIDFactory; import org.teiid.dqp.internal.process.DQPWorkContext; import org.teiid.query.analysis.AnalysisRecord; import org.teiid.query.metadata.QueryMetadataInterface; @@ -113,7 +112,6 @@ CapabilitiesFinder finder = new MultiSourceCapabilitiesFinder(fakeFinder, multiSourceModels); IDGenerator idGenerator = new IDGenerator(); - idGenerator.setDefaultFactory(new IntegerIDFactory()); Properties props = new Properties(); CommandContext context = new CommandContext("0", "test", "user", null, vdb.getName(), vdb.getVersion(), props, false); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestAggregatePushdown.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -69,7 +69,7 @@ String sql = "SELECT a12.intkey AS REGION_NBR, SUM(a11.intnum) AS WJXBFS1 FROM bqt1.smalla AS a11 INNER JOIN bqt2.smalla AS a12 ON a11.stringkey = a12.stringkey WHERE a11.stringkey = 0 GROUP BY a12.intkey"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.exampleBQTCached(), null, capFinder, - new String[] {"SELECT SUM(a11.intnum) FROM bqt1.smalla AS a11 WHERE a11.stringkey = '0' HAVING COUNT(*) > 0", "SELECT a12.intkey FROM bqt2.smalla AS a12 WHERE a12.stringkey = '0' group by a12.intkey"}, TestOptimizer.SHOULD_SUCCEED); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT SUM(a11.intnum) FROM bqt1.smalla AS a11 WHERE a11.stringkey = '0' HAVING COUNT(*) > 0", "SELECT a12.intkey FROM bqt2.smalla AS a12 WHERE a12.stringkey = '0'"}, TestOptimizer.SHOULD_SUCCEED); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -203,7 +203,7 @@ String sql = "select sum(a11.intnum) Profit, (sum(a11.intnum) / sum(a11.floatnum)) WJXBFS2 from bqt1.smalla a11 join bqt2.smallb a12 on a11.intkey=a12.intkey group by a12.intkey"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.exampleBQTCached(), null, capFinder, - new String[] {"SELECT g_0.intkey, SUM(g_0.intnum), SUM(g_0.floatnum) FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey", "SELECT g_0.intkey FROM bqt2.smallb AS g_0 GROUP BY g_0.intkey"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.intkey, SUM(g_0.intnum), SUM(g_0.floatnum) FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey", "SELECT g_0.intkey FROM bqt2.smallb AS g_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -298,7 +298,7 @@ String sql = "SELECT avg(y.e2) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, - new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0 GROUP BY g_0.e3, g_0.e2", "SELECT g_0.e3, g_0.e1, SUM(g_0.e2), COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0", "SELECT g_0.e3, g_0.e1, SUM(g_0.e2), COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -317,7 +317,71 @@ 0 // UnionAll }); } + + @Test public void testAvgAggregateFiltered() throws Exception { + FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); + BasicSourceCapabilities caps = getAggregateCapabilities(); + caps.setCapabilitySupport(Capability.QUERY_ORDERBY, false); + capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ + capFinder.addCapabilities("pm2", caps); //$NON-NLS-1$ + + String sql = "SELECT avg(y.e2) filter (where y.e1 = 1) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ + ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, + new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0", + "SELECT g_0.e3, g_0.e1, g_0.e2 FROM pm2.g1 AS g_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + + TestOptimizer.checkNodeTypes(plan, new int[] { + 2, // Access + 0, // DependentAccess + 0, // DependentSelect + 0, // DependentProject + 0, // DupRemove + 2, // Grouping + 0, // NestedLoopJoinStrategy + 1, // MergeJoinStrategy + 0, // Null + 0, // PlanExecution + 1, // Project + 0, // Select + 0, // Sort + 0 // UnionAll + }); + } + /** + * Agg can only be computed after the join + */ + @Test public void testAvgAggregateFiltered1() throws Exception { + FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); + BasicSourceCapabilities caps = getAggregateCapabilities(); + caps.setCapabilitySupport(Capability.ADVANCED_OLAP, true); + caps.setCapabilitySupport(Capability.QUERY_ORDERBY, false); + capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ + capFinder.addCapabilities("pm2", caps); //$NON-NLS-1$ + + String sql = "SELECT avg(y.e2) filter (where x.e1 = 1) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ + ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, + new String[] {"SELECT g_0.e3, g_0.e2, g_0.e1 FROM pm1.g1 AS g_0", + "SELECT g_0.e3, g_0.e1, g_0.e2 FROM pm2.g1 AS g_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + + TestOptimizer.checkNodeTypes(plan, new int[] { + 2, // Access + 0, // DependentAccess + 0, // DependentSelect + 0, // DependentProject + 0, // DupRemove + 1, // Grouping + 0, // NestedLoopJoinStrategy + 1, // MergeJoinStrategy + 0, // Null + 0, // PlanExecution + 1, // Project + 0, // Select + 0, // Sort + 0 // UnionAll + }); + } + @Test public void testStddevAggregate() throws Exception { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = getAggregateCapabilities(); @@ -328,7 +392,7 @@ String sql = "SELECT stddev_pop(y.e2) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, - new String[] {"SELECT g_0.e3 AS c_0, g_0.e1 AS c_1, COUNT(g_0.e2) AS c_2, SUM(power(g_0.e2, 2)) AS c_3, SUM(g_0.e2) AS c_4 FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1 ORDER BY c_0", "SELECT g_0.e3 AS c_0, g_0.e2 AS c_1 FROM pm1.g1 AS g_0 GROUP BY g_0.e3, g_0.e2 ORDER BY c_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.e3 AS c_0, g_0.e1 AS c_1, COUNT(g_0.e2) AS c_2, SUM(power(g_0.e2, 2)) AS c_3, SUM(g_0.e2) AS c_4 FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1 ORDER BY c_0", "SELECT g_0.e3 AS c_0, g_0.e2 AS c_1 FROM pm1.g1 AS g_0 ORDER BY c_0"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -357,7 +421,7 @@ String sql = "SELECT count(y.e2) from pm1.g1 x, pm2.g1 y where x.e3 = y.e3 group by x.e2, y.e1"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), null, capFinder, - new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0 GROUP BY g_0.e3, g_0.e2", "SELECT g_0.e3, g_0.e1, COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ + new String[] {"SELECT g_0.e3, g_0.e2 FROM pm1.g1 AS g_0", "SELECT g_0.e3, g_0.e1, COUNT(g_0.e2) FROM pm2.g1 AS g_0 GROUP BY g_0.e3, g_0.e1"}, TestOptimizer.ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ //$NON-NLS-2$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -767,8 +831,8 @@ ProcessorPlan plan = helpPlan(sql, metadata, null, capFinder, - new String[] {"SELECT c2, c1, c0 FROM db2model.DB2TABLE", //$NON-NLS-1$ - "SELECT DISTINCT b2, sum(b0) FROM oraclemodel.OraTable GROUP BY b2 ORDER BY b2"}, //$NON-NLS-1$ + new String[] {"SELECT g_0.c2, g_0.c1, g_0.c0 FROM db2model.DB2TABLE AS g_0", //$NON-NLS-1$ + "SELECT g_0.b2 AS c_0, g_0.b0 AS c_1 FROM oraclemodel.OraTable AS g_0 ORDER BY c_0"}, //$NON-NLS-1$ SHOULD_SUCCEED ); checkNodeTypes(plan, new int[] { @@ -777,12 +841,12 @@ 0, // DependentSelect 0, // DependentProject 0, // DupRemove - 1, // Grouping + 2, // Grouping 0, // NestedLoopJoinStrategy 1, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 0, // Select 0, // Sort 0 // UnionAll @@ -815,6 +879,90 @@ }); } + /** + * olap not supported + */ + @Test public void testPushDownOverUnionFiltered() throws Exception { + FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); + BasicSourceCapabilities caps = getAggregateCapabilities(); + capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ + + ProcessorPlan plan = TestOptimizer.helpPlan("select e1, count(*) filter (where e3) from (select e1, e2, e3 from pm1.g1 union all select e1, e2, e3 from pm1.g2) y group by e1", RealMetadataFactory.example1Cached(), null, capFinder, //$NON-NLS-1$ + new String[]{"SELECT g_0.e1, g_0.e3 FROM pm1.g1 AS g_0", "SELECT g_0.e1, g_0.e3 FROM pm1.g2 AS g_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + TestOptimizer.checkNodeTypes(plan, new int[] { + 2, // Access + 0, // DependentAccess + 0, // DependentSelect + 0, // DependentProject + 0, // DupRemove + 1, // Grouping + 0, // NestedLoopJoinStrategy + 0, // MergeJoinStrategy + 0, // Null + 0, // PlanExecution + 1, // Project + 0, // Select + 0, // Sort + 1 // UnionAll + }); + } + + @Test public void testPushDownOverUnionFiltered1() throws Exception { + FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); + BasicSourceCapabilities caps = getAggregateCapabilities(); + caps.setCapabilitySupport(Capability.ADVANCED_OLAP, true); + capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ + + ProcessorPlan plan = TestOptimizer.helpPlan("select e1, count(*) filter (where e3) from (select e1, e2, e3 from pm1.g1 union all select e1, e2, e3 from pm1.g2) y group by e1", RealMetadataFactory.example1Cached(), null, capFinder, //$NON-NLS-1$ + new String[]{"SELECT v_0.c_0, COUNT(*) FILTER(WHERE v_0.c_1) FROM (SELECT g_0.e1 AS c_0, g_0.e3 AS c_1 FROM pm1.g1 AS g_0) AS v_0 GROUP BY v_0.c_0", + "SELECT v_0.c_0, COUNT(*) FILTER(WHERE v_0.c_1) FROM (SELECT g_0.e1 AS c_0, g_0.e3 AS c_1 FROM pm1.g2 AS g_0) AS v_0 GROUP BY v_0.c_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + TestOptimizer.checkNodeTypes(plan, new int[] { + 2, // Access + 0, // DependentAccess + 0, // DependentSelect + 0, // DependentProject + 0, // DupRemove + 1, // Grouping + 0, // NestedLoopJoinStrategy + 0, // MergeJoinStrategy + 0, // Null + 0, // PlanExecution + 1, // Project + 0, // Select + 0, // Sort + 1 // UnionAll + }); + } + + /** + * rand pushdown not supported + */ + @Test public void testPushDownOverUnionFiltered3() throws Exception { + FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); + BasicSourceCapabilities caps = getAggregateCapabilities(); + caps.setCapabilitySupport(Capability.ADVANCED_OLAP, true); + capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ + + ProcessorPlan plan = TestOptimizer.helpPlan("select e1, count(*) filter (where e3 > rand()) from (select e1, e2, e3 from pm1.g1 union all select e1, e2, e3 from pm1.g2) y group by e1", RealMetadataFactory.example1Cached(), null, capFinder, //$NON-NLS-1$ + new String[]{"SELECT g_0.e1, g_0.e3 FROM pm1.g1 AS g_0", "SELECT g_0.e1, g_0.e3 FROM pm1.g2 AS g_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + TestOptimizer.checkNodeTypes(plan, new int[] { + 2, // Access + 0, // DependentAccess + 0, // DependentSelect + 0, // DependentProject + 0, // DupRemove + 1, // Grouping + 0, // NestedLoopJoinStrategy + 0, // MergeJoinStrategy + 0, // Null + 0, // PlanExecution + 1, // Project + 0, // Select + 0, // Sort + 1 // UnionAll + }); + } + @Test public void testPushDownOverUnion1() throws Exception { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = getAggregateCapabilities(); @@ -908,7 +1056,7 @@ capFinder.addCapabilities("pm2", getAggregateCapabilities()); //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan("select max(e2), case when e1 is null then 0 else 1 end from (select e1, e2 from pm1.g1 union all select e1, e2 from pm2.g2) z group by case when e1 is null then 0 else 1 end", RealMetadataFactory.example1Cached(), null, capFinder, //$NON-NLS-1$ - new String[]{"SELECT v_1.c_0, MAX(v_1.c_1) FROM (SELECT CASE WHEN v_0.c_0 IS NULL THEN 0 ELSE 1 END AS c_0, v_0.c_1 FROM (SELECT g_0.e1 AS c_0, g_0.e2 AS c_1 FROM pm1.g1 AS g_0) AS v_0) AS v_1 GROUP BY v_1.c_0", //$NON-NLS-1$ + new String[]{"SELECT v_1.c_1, MAX(v_1.c_0) FROM (SELECT v_0.c_1 AS c_0, CASE WHEN v_0.c_0 IS NULL THEN 0 ELSE 1 END AS c_1 FROM (SELECT g_0.e1 AS c_0, g_0.e2 AS c_1 FROM pm1.g1 AS g_0) AS v_0) AS v_1 GROUP BY v_1.c_1", //$NON-NLS-1$ "SELECT g_0.e1, g_0.e2 FROM pm2.g2 AS g_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access @@ -921,7 +1069,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 3, // Project + 2, // Project 0, // Select 0, // Sort 1 // UnionAll @@ -935,8 +1083,8 @@ capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan("select max(e2), case when e1 is null then 0 else 1 end from (select e1, e2, 1 as part from pm1.g1 union all select e1, e2, 2 as part from pm1.g2) z group by case when e1 is null then 0 else 1 end, part", RealMetadataFactory.example1Cached(), null, capFinder, //$NON-NLS-1$ - new String[]{"SELECT MAX(v_1.c_2), v_1.c_0 FROM (SELECT CASE WHEN v_0.c_0 IS NULL THEN 0 ELSE 1 END AS c_0, v_0.c_1, v_0.c_2 FROM (SELECT g_0.e1 AS c_0, 2 AS c_1, g_0.e2 AS c_2 FROM pm1.g2 AS g_0) AS v_0) AS v_1 GROUP BY v_1.c_0, v_1.c_1", - "SELECT MAX(v_1.c_2), v_1.c_0 FROM (SELECT CASE WHEN v_0.c_0 IS NULL THEN 0 ELSE 1 END AS c_0, v_0.c_1, v_0.c_2 FROM (SELECT g_0.e1 AS c_0, 1 AS c_1, g_0.e2 AS c_2 FROM pm1.g1 AS g_0) AS v_0) AS v_1 GROUP BY v_1.c_0, v_1.c_1"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + new String[]{"SELECT MAX(v_1.c_0), v_1.c_1 FROM (SELECT v_0.c_2 AS c_0, CASE WHEN v_0.c_0 IS NULL THEN 0 ELSE 1 END AS c_1, v_0.c_1 AS c_2 FROM (SELECT g_0.e1 AS c_0, 1 AS c_1, g_0.e2 AS c_2 FROM pm1.g1 AS g_0) AS v_0) AS v_1 GROUP BY v_1.c_1, v_1.c_2", + "SELECT MAX(v_1.c_0), v_1.c_1 FROM (SELECT v_0.c_2 AS c_0, CASE WHEN v_0.c_0 IS NULL THEN 0 ELSE 1 END AS c_1, v_0.c_1 AS c_2 FROM (SELECT g_0.e1 AS c_0, 2 AS c_1, g_0.e2 AS c_2 FROM pm1.g2 AS g_0) AS v_0) AS v_1 GROUP BY v_1.c_1, v_1.c_2"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ TestOptimizer.checkNodeTypes(plan, new int[] { 2, // Access 0, // DependentAccess Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestExpressionsInGroupBy.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestExpressionsInGroupBy.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestExpressionsInGroupBy.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -32,10 +32,7 @@ /** - * expressions in group use lacks robust support in MySQL, PostGres, and Derby Expressions and it's nothing more than syntactic sugar for an inline view, - * a new approach was taken to use inline views rather than a non ANSI group by construct. - * - * Later we can add a connector binding property to support non-select expressions in group by. + * expressions in group use lacks robust support in MySQL, PostGres, and Derby, so a compensation step must be taken to create an inline view */ public class TestExpressionsInGroupBy { @@ -57,7 +54,7 @@ // Plan query ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.exampleBQTCached(), null, capFinder, - new String[] { "SELECT convert(TimestampValue, date), COUNT(*) FROM bqt1.smalla GROUP BY convert(TimestampValue, date)" }, //$NON-NLS-1$ + new String[] { "SELECT v_0.c_0, COUNT(*) FROM (SELECT convert(g_0.TimestampValue, date) AS c_0 FROM bqt1.smalla AS g_0) AS v_0 GROUP BY v_0.c_0" }, //$NON-NLS-1$ true); TestOptimizer.checkNodeTypes(plan, TestOptimizer.FULL_PUSHDOWN); } @@ -81,7 +78,7 @@ // Plan query ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.exampleBQTCached(), null, capFinder, - new String[] { "SELECT convert(TimestampValue, date), COUNT(*) FROM bqt1.smalla GROUP BY convert(TimestampValue, date)" }, //$NON-NLS-1$ + new String[] { "SELECT v_0.c_0, COUNT(*) FROM (SELECT convert(g_0.TimestampValue, date) AS c_0 FROM bqt1.smalla AS g_0) AS v_0 GROUP BY v_0.c_0" }, //$NON-NLS-1$ true); TestOptimizer.checkNodeTypes(plan, TestOptimizer.FULL_PUSHDOWN); } @@ -105,7 +102,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 0, // Select 0, // Sort 0 // UnionAll @@ -143,7 +140,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 0, // Select 0, // Sort 0 // UnionAll @@ -181,7 +178,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 0, // Select 0, // Sort 0 // UnionAll @@ -213,7 +210,7 @@ } /** - * Without inline view support the agg is not pushed down + * Without inline view support or functions in group by the agg is not pushed down */ @Test public void testFunctionInGroupBy() { String sql = "SELECT sum (IntKey), case when IntKey>=5000 then '5000 +' else '0-999' end " + //$NON-NLS-1$ @@ -226,6 +223,7 @@ caps.setCapabilitySupport(Capability.QUERY_CASE, true); caps.setCapabilitySupport(Capability.QUERY_SEARCHED_CASE, true); caps.setCapabilitySupport(Capability.QUERY_AGGREGATES_SUM, true); + caps.setCapabilitySupport(Capability.QUERY_GROUP_BY, true); caps.setCapabilitySupport(Capability.CRITERIA_COMPARE_ORDERED, true); caps.setCapabilitySupport(Capability.CRITERIA_COMPARE_EQ, true); capFinder.addCapabilities("BQT1", caps); //$NON-NLS-1$ @@ -233,7 +231,7 @@ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.exampleBQTCached(), null, capFinder, - new String[] {"SELECT CASE WHEN BQT1.SmallA.IntKey >= 5000 THEN '5000 +' ELSE '0-999' END, BQT1.SmallA.IntKey FROM BQT1.SmallA"}, //$NON-NLS-1$ + new String[] {"SELECT BQT1.SmallA.IntKey FROM BQT1.SmallA"}, //$NON-NLS-1$ TestOptimizer.SHOULD_SUCCEED ); TestOptimizer.checkNodeTypes(plan, new int[] { @@ -254,6 +252,60 @@ }); } + @Test public void testFunctionInGroupBy1() { + String sql = "SELECT sum (IntKey), case when IntKey>=5000 then '5000 +' else '0-999' end " + //$NON-NLS-1$ + "FROM BQT1.SmallA GROUP BY case when IntKey>=5000 then '5000 +' else '0-999' end"; //$NON-NLS-1$ + + // Plan query + FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); + BasicSourceCapabilities caps = new BasicSourceCapabilities(); + caps.setCapabilitySupport(Capability.QUERY_SELECT_EXPRESSION, true); + caps.setCapabilitySupport(Capability.QUERY_CASE, true); + caps.setCapabilitySupport(Capability.QUERY_SEARCHED_CASE, true); + caps.setCapabilitySupport(Capability.QUERY_AGGREGATES_SUM, true); + caps.setCapabilitySupport(Capability.QUERY_FUNCTIONS_IN_GROUP_BY, true); + caps.setCapabilitySupport(Capability.QUERY_GROUP_BY, true); + caps.setCapabilitySupport(Capability.CRITERIA_COMPARE_ORDERED, true); + caps.setCapabilitySupport(Capability.CRITERIA_COMPARE_EQ, true); + capFinder.addCapabilities("BQT1", caps); //$NON-NLS-1$ + + ProcessorPlan plan = TestOptimizer.helpPlan(sql, + RealMetadataFactory.exampleBQTCached(), + null, capFinder, + new String[] {"SELECT SUM(BQT1.SmallA.IntKey), CASE WHEN BQT1.SmallA.IntKey >= 5000 THEN '5000 +' ELSE '0-999' END FROM BQT1.SmallA GROUP BY CASE WHEN BQT1.SmallA.IntKey >= 5000 THEN '5000 +' ELSE '0-999' END"}, //$NON-NLS-1$ + TestOptimizer.SHOULD_SUCCEED ); + + TestOptimizer.checkNodeTypes(plan, TestOptimizer.FULL_PUSHDOWN); + } + + @Test public void testFunctionInGroupBy2() { + String sql = "SELECT sum (IntKey), case when IntKey>=5000 then '5000 +' else '0-999' end " + //$NON-NLS-1$ + "FROM BQT1.SmallA GROUP BY case when IntKey>=5000 then '5000 +' else '0-999' end"; //$NON-NLS-1$ + + // Plan query + FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); + BasicSourceCapabilities caps = new BasicSourceCapabilities(); + caps.setCapabilitySupport(Capability.QUERY_SELECT_EXPRESSION, true); + caps.setCapabilitySupport(Capability.QUERY_CASE, true); + caps.setCapabilitySupport(Capability.QUERY_SEARCHED_CASE, true); + caps.setCapabilitySupport(Capability.QUERY_AGGREGATES_SUM, true); + caps.setCapabilitySupport(Capability.QUERY_FROM_INLINE_VIEWS, true); + caps.setCapabilitySupport(Capability.QUERY_GROUP_BY, true); + caps.setCapabilitySupport(Capability.QUERY_FROM_GROUP_ALIAS, true); + caps.setCapabilitySupport(Capability.CRITERIA_COMPARE_ORDERED, true); + caps.setCapabilitySupport(Capability.CRITERIA_COMPARE_EQ, true); + capFinder.addCapabilities("BQT1", caps); //$NON-NLS-1$ + + ProcessorPlan plan = TestOptimizer.helpPlan(sql, + RealMetadataFactory.exampleBQTCached(), + null, capFinder, + new String[] {"SELECT SUM(v_0.c_1), v_0.c_0 FROM (SELECT CASE WHEN g_0.IntKey >= 5000 THEN '5000 +' ELSE '0-999' END AS c_0, g_0.IntKey AS c_1 FROM BQT1.SmallA AS g_0) AS v_0 GROUP BY v_0.c_0"}, //$NON-NLS-1$ + TestOptimizer.SHOULD_SUCCEED ); + + TestOptimizer.checkNodeTypes(plan, TestOptimizer.FULL_PUSHDOWN); + } + + /** * Test what happens when we have a CASE in the GROUP BY and source has aggregate capability but * does not have CASE capability. Should not be able to push down GROUP BY. @@ -291,7 +343,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 0, // Select 0, // Sort 0 // UnionAll @@ -336,7 +388,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 1, // Select 0, // Sort 0 // UnionAll @@ -378,7 +430,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 0, // Select 0, // Sort 0 // UnionAll @@ -410,7 +462,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 1, // Select 0, // Sort 0 // UnionAll @@ -442,7 +494,7 @@ 0, // MergeJoinStrategy 0, // Null 0, // PlanExecution - 2, // Project + 1, // Project 0, // Select 0, // Sort 0 // UnionAll Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestInlineView.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestInlineView.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestInlineView.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -25,6 +25,7 @@ import static org.teiid.query.optimizer.TestOptimizer.*; import java.util.ArrayList; +import java.util.Arrays; import java.util.HashSet; import java.util.List; import java.util.Set; @@ -94,10 +95,8 @@ String userQuery = "select q1.a from (select count(bqt1.smalla.intkey) as a, bqt1.smalla.intkey from bqt1.smalla group by bqt1.smalla.intkey) q1 left outer join bqt1.smallb on q1.a = bqt1.smallb.intkey where q1.intkey = 1"; //$NON-NLS-1$ String optimizedQuery = "SELECT v_0.c_0 FROM (SELECT COUNT(g_0.intkey) AS c_0 FROM bqt1.smalla AS g_0 WHERE g_0.intkey = 1 GROUP BY g_0.intkey) AS v_0 LEFT OUTER JOIN bqt1.smallb AS g_1 ON v_0.c_0 = g_1.intkey"; //$NON-NLS-1$ - List expectedResults = new ArrayList(); - List row1 = new ArrayList(); - row1.add(new Integer(1)); - expectedResults.add(row1); + List<List<?>> expectedResults = new ArrayList<List<?>>(); + expectedResults.add(Arrays.asList(1)); Set<String> sourceQueries = new HashSet<String>(); sourceQueries.add("oracle"); //$NON-NLS-1$ @@ -116,10 +115,8 @@ "where bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.intkey = 1 and bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = bqt1.smallb.intkey"; //$NON-NLS-1$ String optimizedQuery = "SELECT v_0.c_0 FROM (SELECT COUNT(g_0.intkey) AS c_0 FROM bqt1.smalla AS g_0 WHERE g_0.intkey = 1 GROUP BY g_0.intkey) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey"; //$NON-NLS-1$ - List expectedResults = new ArrayList(); - List row1 = new ArrayList(); - row1.add(new Integer(1)); - expectedResults.add(row1); + List<List<?>> expectedResults = new ArrayList<List<?>>(); + expectedResults.add(Arrays.asList(1)); Set<String> sourceQueries = new HashSet<String>(); sourceQueries.add("oracle"); //$NON-NLS-1$ @@ -134,12 +131,10 @@ public static InlineViewCase createInlineViewWithDistinctAndOrderBy() throws Exception { String userQuery = "select Q1.a from (select distinct count(bqt1.smalla.intkey) as a, bqt1.smalla.intkey from bqt1.smalla group by bqt1.smalla.intkey order by bqt1.smalla.intkey) q1 inner join bqt1.smallb as q2 on q1.intkey = q2.intkey where q1.a = 1 and q1.a + q1.intkey = 2"; //$NON-NLS-1$ - String optimizedQuery = "SELECT v_0.c_1 FROM (SELECT g_0.intkey AS c_0, COUNT(g_0.intkey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey HAVING ((COUNT(g_0.intkey) + g_0.intkey) = 2) AND (COUNT(g_0.intkey) = 1)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey"; //$NON-NLS-1$ + String optimizedQuery = "SELECT v_0.c_1 FROM (SELECT g_0.intkey AS c_0, COUNT(g_0.intkey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey HAVING (COUNT(g_0.intkey) = 1) AND ((COUNT(g_0.intkey) + g_0.intkey) = 2)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey"; //$NON-NLS-1$ - List expectedResults = new ArrayList(); - List row1 = new ArrayList(); - row1.add(new Integer(1)); - expectedResults.add(row1); + List<List<?>> expectedResults = new ArrayList<List<?>>(); + expectedResults.add(Arrays.asList(1)); Set<String> sourceQueries = new HashSet<String>(); sourceQueries.add("oracle"); //$NON-NLS-1$ @@ -157,7 +152,7 @@ String userQuery = "select q1.A from (select count(intkey) as a, intkey, stringkey from vqt.smalla group by intkey, stringkey) q1 inner join vqt.smallb as q2 on q1.intkey = q2.a12345 where q1.a = 2"; //$NON-NLS-1$ String optimizedQuery = "SELECT v_0.c_1 FROM (SELECT g_0.IntKey AS c_0, COUNT(g_0.IntKey) AS c_1 FROM BQT1.SmallA AS g_0 GROUP BY g_0.IntKey, g_0.StringKey HAVING COUNT(g_0.IntKey) = 2) AS v_0, BQT1.SmallA AS g_1 WHERE v_0.c_0 = Concat(g_1.StringKey, g_1.StringNum)"; //$NON-NLS-1$ - List expectedResults = new ArrayList(); + List<List<?>> expectedResults = new ArrayList<List<?>>(); Set<String> sourceQueries = new HashSet<String>(); sourceQueries.add("oracle"); //$NON-NLS-1$ @@ -172,12 +167,10 @@ public static InlineViewCase createInlineViewWithOuterOrderAndGroup() throws Exception { String userQuery = "select count(Q1.a) b from (select distinct count(bqt1.smalla.intkey) as a, bqt1.smalla.intkey from bqt1.smalla group by bqt1.smalla.intkey order by bqt1.smalla.intkey) q1 inner join bqt1.smallb as q2 on q1.intkey = q2.intkey where q1.a = 1 and q1.a + q1.intkey = 2 group by Q1.a order by b"; //$NON-NLS-1$ - String optimizedQuery = "SELECT COUNT(v_0.c_1) AS c_0 FROM (SELECT g_0.intkey AS c_0, COUNT(g_0.intkey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey HAVING ((COUNT(g_0.intkey) + g_0.intkey) = 2) AND (COUNT(g_0.intkey) = 1)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey GROUP BY v_0.c_1 ORDER BY c_0"; //$NON-NLS-1$ + String optimizedQuery = "SELECT COUNT(v_0.c_1) AS c_0 FROM (SELECT g_0.intkey AS c_0, COUNT(g_0.intkey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.intkey HAVING (COUNT(g_0.intkey) = 1) AND ((COUNT(g_0.intkey) + g_0.intkey) = 2)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey GROUP BY v_0.c_1 ORDER BY c_0"; //$NON-NLS-1$ - List expectedResults = new ArrayList(); - List row1 = new ArrayList(); - row1.add(new Integer(1)); - expectedResults.add(row1); + List<List<?>> expectedResults = new ArrayList<List<?>>(); + expectedResults.add(Arrays.asList(1)); Set<String> sourceQueries = new HashSet<String>(); sourceQueries.add("oracle"); //$NON-NLS-1$ @@ -192,15 +185,11 @@ public static InlineViewCase crateInlineViewsInUnions() throws Exception { String userQuery = "select q1.a from (select count(bqt1.smalla.intkey) as a, bqt1.smalla.intkey from bqt1.smalla group by bqt1.smalla.intkey) q1 left outer join bqt1.smallb on q1.a = bqt1.smallb.intkey where q1.intkey = 1 union all (select count(Q1.a) b from (select distinct count(bqt1.smalla.intkey) as a, bqt1.smalla.intkey from bqt1.smalla group by bqt1.smalla.intkey order by bqt1.smalla.intkey) q1 inner join bqt1.smallb as q2 on q1.intkey = q2.intkey where q1.a = 1 and q1.a + q1.intkey = 2 group by Q1.a order by b)"; //$NON-NLS-1$ - String optimizedQuery = "SELECT v_1.c_0 FROM (SELECT COUNT(g_2.intkey) AS c_0 FROM bqt1.smalla AS g_2 WHERE g_2.intkey = 1 GROUP BY g_2.intkey) AS v_1 LEFT OUTER JOIN bqt1.smallb AS g_3 ON v_1.c_0 = g_3.intkey UNION ALL SELECT COUNT(v_0.c_1) AS c_0 FROM (SELECT g_0.IntKey AS c_0, COUNT(g_0.IntKey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.IntKey HAVING ((COUNT(g_0.IntKey) + g_0.IntKey) = 2) AND (COUNT(g_0.IntKey) = 1)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey GROUP BY v_0.c_1"; //$NON-NLS-1$ + String optimizedQuery = "SELECT v_1.c_0 FROM (SELECT COUNT(g_2.intkey) AS c_0 FROM bqt1.smalla AS g_2 WHERE g_2.intkey = 1 GROUP BY g_2.intkey) AS v_1 LEFT OUTER JOIN bqt1.smallb AS g_3 ON v_1.c_0 = g_3.intkey UNION ALL SELECT COUNT(v_0.c_1) AS c_0 FROM (SELECT g_0.IntKey AS c_0, COUNT(g_0.IntKey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.IntKey HAVING (COUNT(g_0.IntKey) = 1) AND ((COUNT(g_0.IntKey) + g_0.IntKey) = 2)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey GROUP BY v_0.c_1"; //$NON-NLS-1$ - List expectedResults = new ArrayList(); - List row1 = new ArrayList(); - row1.add(new Integer(1)); - expectedResults.add(row1); - List row2 = new ArrayList(); - row2.add(new Integer(1)); - expectedResults.add(row2); + List<List<?>> expectedResults = new ArrayList<List<?>>(); + expectedResults.add(Arrays.asList(1)); + expectedResults.add(Arrays.asList(2)); Set<String> sourceQueries = new HashSet<String>(); sourceQueries.add("oracle"); //$NON-NLS-1$ @@ -217,15 +206,11 @@ public static InlineViewCase createUnionInInlineView() throws Exception{ String userQuery = "select t1.intkey from (select case when q1.a=1 then 2 else 1 end as a from (select count(bqt1.smalla.intkey) as a, bqt1.smalla.intkey from bqt1.smalla group by bqt1.smalla.intkey) q1 left outer join bqt1.smallb on q1.a = bqt1.smallb.intkey where q1.intkey = 1 union all (select count(Q1.a) b from (select distinct count(bqt1.smalla.intkey) as a, bqt1.smalla.intkey from bqt1.smalla group by bqt1.smalla.intkey order by bqt1.smalla.intkey) q1 inner join bqt1.smallb as q2 on q1.intkey = q2.intkey where q1.a = 1 and q1.a + q1.intkey = 2 group by Q1.a order by b)) as q3, bqt1.smallb as t1 where q3.a = t1.intkey order by t1.intkey"; //$NON-NLS-1$ - String optimizedQuery = "SELECT g_4.intkey AS c_0 FROM (SELECT CASE WHEN v_1.c_0 = 1 THEN 2 ELSE 1 END AS c_0 FROM (SELECT COUNT(g_2.intkey) AS c_0 FROM bqt1.smalla AS g_2 WHERE g_2.intkey = 1 GROUP BY g_2.intkey) AS v_1 LEFT OUTER JOIN bqt1.smallb AS g_3 ON v_1.c_0 = g_3.intkey UNION ALL SELECT COUNT(v_0.c_1) AS c_0 FROM (SELECT g_0.IntKey AS c_0, COUNT(g_0.IntKey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.IntKey HAVING ((COUNT(g_0.IntKey) + g_0.IntKey) = 2) AND (COUNT(g_0.IntKey) = 1)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey GROUP BY v_0.c_1) AS v_2, bqt1.smallb AS g_4 WHERE v_2.c_0 = g_4.intkey ORDER BY c_0"; //$NON-NLS-1$ + String optimizedQuery = "SELECT g_4.intkey AS c_0 FROM (SELECT CASE WHEN v_1.c_0 = 1 THEN 2 ELSE 1 END AS c_0 FROM (SELECT COUNT(g_2.intkey) AS c_0 FROM bqt1.smalla AS g_2 WHERE g_2.intkey = 1 GROUP BY g_2.intkey) AS v_1 LEFT OUTER JOIN bqt1.smallb AS g_3 ON v_1.c_0 = g_3.intkey UNION ALL SELECT COUNT(v_0.c_1) AS c_0 FROM (SELECT g_0.IntKey AS c_0, COUNT(g_0.IntKey) AS c_1 FROM bqt1.smalla AS g_0 GROUP BY g_0.IntKey HAVING (COUNT(g_0.IntKey) = 1) AND ((COUNT(g_0.IntKey) + g_0.IntKey) = 2)) AS v_0, bqt1.smallb AS g_1 WHERE v_0.c_0 = g_1.intkey GROUP BY v_0.c_1) AS v_2, bqt1.smallb AS g_4 WHERE v_2.c_0 = g_4.intkey ORDER BY c_0"; //$NON-NLS-1$ - List expectedResults = new ArrayList(); - List row1 = new ArrayList(); - row1.add(new Integer(1)); - expectedResults.add(row1); - List row2 = new ArrayList(); - row2.add(new Integer(2)); - expectedResults.add(row2); + List<List<?>> expectedResults = new ArrayList<List<?>>(); + expectedResults.add(Arrays.asList(1)); + expectedResults.add(Arrays.asList(2)); Set<String> sourceQueries = new HashSet<String>(); sourceQueries.add("oracle"); //$NON-NLS-1$ Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestLimit.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestLimit.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestLimit.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -25,8 +25,7 @@ import java.util.Arrays; import java.util.List; -import junit.framework.TestCase; - +import org.junit.Test; import org.teiid.core.types.DataTypeManager; import org.teiid.metadata.Column; import org.teiid.metadata.MetadataStore; @@ -59,7 +58,7 @@ import org.teiid.query.unittest.RealMetadataFactory; @SuppressWarnings("nls") -public class TestLimit extends TestCase { +public class TestLimit { private static final int[] FULL_PUSHDOWN = new int[] { 1, // Access @@ -97,10 +96,6 @@ UnionAllNode.class }; - public TestLimit(String name) { - super(name); - } - private static TransformationMetadata exampleMetadata() { MetadataStore metadataStore = new MetadataStore(); // Create models @@ -157,7 +152,7 @@ return RealMetadataFactory.createTransformationMetadata(metadataStore, "example"); } - public void testLimit() { + @Test public void testLimit() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ @@ -189,7 +184,7 @@ }, NODE_TYPES); } - public void testLimitPushdown() { + @Test public void testLimitPushdown() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -205,7 +200,7 @@ TestOptimizer.checkNodeTypes(plan, FULL_PUSHDOWN, NODE_TYPES); } - public void testLimitWithOffset() { + @Test public void testLimitWithOffset() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ @@ -236,7 +231,7 @@ }, NODE_TYPES); } - public void testPushedLimitWithOffset() throws Exception { + @Test public void testPushedLimitWithOffset() throws Exception { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -268,7 +263,7 @@ }, NODE_TYPES); } - public void testLimitWithOffsetFullyPushed() { + @Test public void testLimitWithOffsetFullyPushed() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -285,7 +280,7 @@ TestOptimizer.checkNodeTypes(plan, FULL_PUSHDOWN, NODE_TYPES); } - public void testSort() { + @Test public void testSort() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ @@ -316,7 +311,7 @@ }, NODE_TYPES); } - public void testSortPushed() { + @Test public void testSortPushed() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.QUERY_ORDERBY, true); @@ -349,7 +344,7 @@ }, NODE_TYPES); } - public void testSortPushedWithLimit() { + @Test public void testSortPushedWithLimit() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.QUERY_ORDERBY, true); @@ -367,7 +362,7 @@ TestOptimizer.checkNodeTypes(plan, FULL_PUSHDOWN, NODE_TYPES); } - public void testSortUnderLimitNotRemoved() { + @Test public void testSortUnderLimitNotRemoved() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); // pm3 model supports order by @@ -400,7 +395,7 @@ } //TODO: there is a redundent project node here - public void testSortAboveLimitNotPushed() { + @Test public void testSortAboveLimitNotPushed() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.QUERY_ORDERBY, true); @@ -433,7 +428,7 @@ }, NODE_TYPES); } - public void testLimitNotPushedWithUnion() { + @Test public void testLimitNotPushedWithUnion() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -466,7 +461,7 @@ }, NODE_TYPES); } - public void testLimitNotPushedWithDupRemove() { + @Test public void testLimitNotPushedWithDupRemove() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -499,7 +494,7 @@ }, NODE_TYPES); } - public void testLimitPushedWithUnionAll() { + @Test public void testLimitPushedWithUnionAll() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -532,7 +527,7 @@ }, NODE_TYPES); } - public void testLimitWithOffsetPushedWithUnion() throws Exception { + @Test public void testLimitWithOffsetPushedWithUnion() throws Exception { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -567,7 +562,7 @@ }, NODE_TYPES); } - public void testLimitNotPushedWithUnionOrderBy() { + @Test public void testLimitNotPushedWithUnionOrderBy() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -601,7 +596,7 @@ TestOptimizer.checkNodeTypes(plan, new int[] {1}, new Class[]{DupRemoveSortNode.class}); } - public void testCombinedLimits() throws Exception { + @Test public void testCombinedLimits() throws Exception { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -635,7 +630,7 @@ }, NODE_TYPES); } - public void testCombinedLimitsWithOffset() throws Exception { + @Test public void testCombinedLimitsWithOffset() throws Exception { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -654,7 +649,7 @@ TestOptimizer.checkNodeTypes(plan, FULL_PUSHDOWN, NODE_TYPES); } - public void testInlineView() { + @Test public void testInlineView() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); //caps.setCapabilitySupport(SourceCapabilities.QUERY_ORDERBY, true); @@ -682,7 +677,7 @@ * * @since 4.3 */ - public void testInlineViewAboveLimitNotMerged() { + @Test public void testInlineViewAboveLimitNotMerged() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.QUERY_ORDERBY, true); @@ -706,7 +701,7 @@ * since there is no order by with the nested limit, the criteria can be pushed through * */ - public void testCriteriaPushedUnderLimit() { + @Test public void testCriteriaPushedUnderLimit() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.ROW_LIMIT, true); @@ -724,7 +719,7 @@ TestOptimizer.checkNodeTypes(plan, FULL_PUSHDOWN, NODE_TYPES); } - public void testInlineViewJoin() { + @Test public void testInlineViewJoin() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); caps.setCapabilitySupport(Capability.QUERY_UNION, true); @@ -768,7 +763,7 @@ }); } - public void testDontPushSelectWithOrderedLimit() { + @Test public void testDontPushSelectWithOrderedLimit() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = TestOptimizer.getTypicalCapabilities(); capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ @@ -798,7 +793,7 @@ }, NODE_TYPES); } - public void testDontPushSelectWithOrderedLimit1() { + @Test public void testDontPushSelectWithOrderedLimit1() { FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = TestOptimizer.getTypicalCapabilities(); capFinder.addCapabilities("pm1", caps); //$NON-NLS-1$ @@ -828,7 +823,7 @@ }, NODE_TYPES); } - public void testLimitWithNoAccessNode() { + @Test public void testLimitWithNoAccessNode() { String sql = "select 1 limit 1";//$NON-NLS-1$ String[] expectedSql = new String[] {}; @@ -856,7 +851,7 @@ /** * Note here that the criteria made it to the having clause */ - public void testAggregateCriteriaOverUnSortedLimit() { + @Test public void testAggregateCriteriaOverUnSortedLimit() { String sql = "select a from (SELECT MAX(e2) as a FROM pm1.g1 GROUP BY e2 LIMIT 1) x where a = 0"; //$NON-NLS-1$ FakeCapabilitiesFinder capFinder = new FakeCapabilitiesFinder(); BasicSourceCapabilities caps = new BasicSourceCapabilities(); @@ -874,7 +869,7 @@ TestOptimizer.checkNodeTypes(plan, FULL_PUSHDOWN, NODE_TYPES); } - public void testSortWithLimitInlineView() { + @Test public void testSortWithLimitInlineView() { String sql = "select e1 from (select pm1.g1.e1, pm1.g1.e2 from pm1.g1 order by pm1.g1.e1, pm1.g1.e2 limit 1) x"; //$NON-NLS-1$ ProcessorPlan plan = TestOptimizer.helpPlan(sql, RealMetadataFactory.example1Cached(), new String[] {"SELECT g_0.e1 AS c_0 FROM pm1.g1 AS g_0 ORDER BY c_0, g_0.e2"}); //$NON-NLS-1$ Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/TestOptimizer.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -6286,7 +6286,7 @@ String sql = "select a.e1 from (select 1 e1) a, (select e1, 1 as a, x from (select e1, CASE WHEN e1 = 'a' THEN e2 ELSE e3 END as x from pm1.g2) y group by e1, x) b where a.e1 = b.x"; //$NON-NLS-1$ ProcessorPlan plan = helpPlan(sql, metadata, null, capFinder, - new String[] {"SELECT v_1.c_0 FROM (SELECT v_0.c_1 AS c_0 FROM (SELECT g_0.e1 AS c_0, CASE WHEN g_0.e1 = 'a' THEN g_0.e2 ELSE g_0.e3 END AS c_1 FROM pm1.g2 AS g_0 WHERE CASE WHEN g_0.e1 = 'a' THEN g_0.e2 ELSE g_0.e3 END IN (<dependent values>)) AS v_0 GROUP BY v_0.c_0, v_0.c_1) AS v_1 ORDER BY c_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + new String[] {"SELECT v_0.c_0 FROM (SELECT CASE WHEN g_0.e1 = 'a' THEN g_0.e2 ELSE g_0.e3 END AS c_0 FROM pm1.g2 AS g_0 GROUP BY g_0.e1, CASE WHEN g_0.e1 = 'a' THEN g_0.e2 ELSE g_0.e3 END) AS v_0 WHERE v_0.c_0 IN (<dependent values>) ORDER BY c_0"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ checkNodeTypes(plan, new int[] { 0, // Access Modified: trunk/engine/src/test/java/org/teiid/query/optimizer/xml/TestXMLPlanner.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/optimizer/xml/TestXMLPlanner.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/optimizer/xml/TestXMLPlanner.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -30,7 +30,6 @@ import org.teiid.core.TeiidComponentException; import org.teiid.core.TeiidProcessingException; import org.teiid.core.id.IDGenerator; -import org.teiid.core.id.IntegerIDFactory; import org.teiid.core.types.DataTypeManager; import org.teiid.metadata.MetadataStore; import org.teiid.metadata.Schema; @@ -689,7 +688,6 @@ public static XMLPlan preparePlan(Command command, QueryMetadataInterface metadata, CapabilitiesFinder capFinder, CommandContext context) throws QueryPlannerException, QueryMetadataException, TeiidComponentException { IDGenerator idGenerator = new IDGenerator(); - idGenerator.setDefaultFactory(new IntegerIDFactory()); AnalysisRecord analysis = new AnalysisRecord(false, DEBUG); try { if (DEBUG) { Modified: trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/parser/TestParser.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -117,6 +117,7 @@ import org.teiid.query.sql.symbol.TestCaseExpression; import org.teiid.query.sql.symbol.TestSearchedCaseExpression; import org.teiid.query.sql.symbol.TextLine; +import org.teiid.query.sql.symbol.WindowFunction; import org.teiid.query.sql.symbol.XMLAttributes; import org.teiid.query.sql.symbol.XMLElement; import org.teiid.query.sql.symbol.XMLForest; @@ -1413,28 +1414,12 @@ @Test public void testFailAliasInHaving() { helpException("SELECT a FROM m.g GROUP BY a, b AS x"); //$NON-NLS-1$ } - - /** SELECT a FROM m.g GROUP BY count(a) */ - @Test public void testFailAggregateInGroupBy() { - helpException("SELECT a FROM m.g GROUP BY count(a)"); //$NON-NLS-1$ - } - - @Test public void testExceptionLength() { + @Test(expected=QueryParserException.class) public void testExceptionLength() throws Exception { String sql = "SELECT * FROM Customer where Customer.Name = (select lastname from CUSTOMER where acctid = 9"; ////$NON-NLS-1$ - try { - QueryParser.getQueryParser().parseCommand(sql); - fail("Expected exception for parsing " + sql); //$NON-NLS-1$ - } catch (TeiidException e) { - //e.printStackTrace(); - //if (e.getMessage().length() > 1000) { - // fail("Expected max length of message 1000; but received " + e.getMessage().length()); //$NON-NLS-1$ - //} - } + QueryParser.getQueryParser().parseCommand(sql); } - - @Test public void testFunctionOfAggregates() { GroupSymbol g = new GroupSymbol("m.g"); //$NON-NLS-1$ From from = new From(); @@ -6425,14 +6410,6 @@ } } - @Test public void testExceptionMessageWithoutLocation() { - try { - QueryParser.getQueryParser().parseCommand("SELECT COUNT(*) FROM a WHERE COUNT(*) > 1"); //$NON-NLS-1$ - } catch(QueryParserException e) { - assertTrue(e.getMessage().startsWith("Parsing error: Aggregate expressions are allowed only as top level functions in the SELECT and HAVING clauses.")); //$NON-NLS-1$ - } - } - @Test public void testEmptyOuterJoinCriteria() { helpException("select a from b left outer join c on ()"); //$NON-NLS-1$ } @@ -6852,5 +6829,34 @@ query.setSelect(new Select(Arrays.asList(new ExpressionSymbol("foo", new Reference(0))))); helpTest(sql, "SELECT ?", query); } + + @Test public void testNonReserved() throws Exception { + String sql = "select count"; + Query query = new Query(); + query.setSelect(new Select(Arrays.asList(new ElementSymbol("count")))); + helpTest(sql, "SELECT count", query); + } + + @Test public void testAggFilter() throws Exception { + String sql = "select count(*) filter (where x = 1) from g"; + Query query = new Query(); + AggregateSymbol aggregateSymbol = new AggregateSymbol("count", AggregateSymbol.Type.COUNT.name(), false, null); + aggregateSymbol.setCondition(new CompareCriteria(new ElementSymbol("x"), CompareCriteria.EQ, new Constant(1))); + query.setSelect(new Select(Arrays.asList(aggregateSymbol))); + query.setFrom(new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("g"))))); + helpTest(sql, "SELECT COUNT(*) FILTER(WHERE x = 1) FROM g", query); + } + + @Test public void testWindowFunction() throws Exception { + String sql = "select row_number() over (partition by x order by y) from g"; + Query query = new Query(); + WindowFunction wf = new WindowFunction(); + wf.setFunction(new AggregateSymbol("expr", "ROW_NUMBER", false, null)); + wf.setPartition(new ArrayList<Expression>(Arrays.asList(new ElementSymbol("x")))); + wf.setOrderBy(new OrderBy(Arrays.asList(new ElementSymbol("y")))); + query.setSelect(new Select(Arrays.asList(new ExpressionSymbol("x", wf)))); + query.setFrom(new From(Arrays.asList(new UnaryFromClause(new GroupSymbol("g"))))); + helpTest(sql, "SELECT ROW_NUMBER() OVER (PARTITION BY x ORDER BY y) FROM g", query); + } } Modified: trunk/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/processor/TestAggregateProcessing.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -266,12 +266,12 @@ capFinder.addCapabilities("pm2", bac); //$NON-NLS-1$ HardcodedDataManager dataManager = new HardcodedDataManager(); - dataManager.addData("SELECT v_0.c_1, v_0.c_0, COUNT(*), MAX(v_0.c_2) FROM (SELECT g_0.e2 AS c_0, g_0.e1 AS c_1, g_0.e3 AS c_2 FROM pm1.g1 AS g_0) AS v_0 GROUP BY v_0.c_0, v_0.c_1", //$NON-NLS-1$ + dataManager.addData("SELECT v_0.c_0, v_0.c_1, COUNT(*), MAX(v_0.c_2) FROM (SELECT g_0.e1 AS c_0, g_0.e2 AS c_1, g_0.e3 AS c_2 FROM pm1.g1 AS g_0) AS v_0 GROUP BY v_0.c_0, v_0.c_1", //$NON-NLS-1$ new List[] { Arrays.asList("2", Integer.valueOf(2), Integer.valueOf(2), Boolean.FALSE), //$NON-NLS-1$ Arrays.asList("1", Integer.valueOf(1), Integer.valueOf(3), Boolean.TRUE), //$NON-NLS-1$ }); - dataManager.addData("SELECT v_0.c_1, v_0.c_0, COUNT(*), MAX(v_0.c_2) FROM (SELECT g_0.e2 AS c_0, convert(g_0.e2, string) AS c_1, g_0.e3 AS c_2 FROM pm2.g2 AS g_0 ORDER BY c_1 LIMIT 10) AS v_0 GROUP BY v_0.c_0, v_0.c_1", //$NON-NLS-1$ + dataManager.addData("SELECT v_0.c_0, v_0.c_1, COUNT(*), MAX(v_0.c_2) FROM (SELECT convert(g_0.e2, string) AS c_0, g_0.e2 AS c_1, g_0.e3 AS c_2 FROM pm2.g2 AS g_0 ORDER BY c_0 LIMIT 10) AS v_0 GROUP BY v_0.c_0, v_0.c_1", //$NON-NLS-1$ new List[] { Arrays.asList("1", Integer.valueOf(1), Integer.valueOf(4), Boolean.FALSE), //$NON-NLS-1$ }); @@ -360,9 +360,9 @@ List[] expected = new List[] { Arrays.asList(null, "cc"), Arrays.asList(0.0, "bb"), - Arrays.asList(1.0, null), - Arrays.asList(2.0, "aa"), - Arrays.asList(7.0, "aa") + Arrays.asList(2.0, null), + Arrays.asList(21.0, "aa"), + Arrays.asList(24.0, "aa") }; // Construct data manager with data @@ -399,5 +399,25 @@ // Run query helpProcess(plan, cc, dataManager, expected); } + + @Test public void testAggFilter() throws Exception { + // Create query + String sql = "SELECT e2, count(*) filter (where e3) from pm1.g1 group by e2 order by e2"; //$NON-NLS-1$ + // Create expected results + List[] expected = new List[] { + Arrays.asList(0, 0), + Arrays.asList(1, 1), + Arrays.asList(2, 0), + Arrays.asList(3, 1), + }; + + // Construct data manager with data + FakeDataManager dataManager = new FakeDataManager(); + sampleData1(dataManager); + + ProcessorPlan plan = helpGetPlan(sql, RealMetadataFactory.example1Cached()); + helpProcess(plan, dataManager, expected); + } + } Modified: trunk/engine/src/test/java/org/teiid/query/processor/TestProcessor.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/TestProcessor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/processor/TestProcessor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -360,7 +360,7 @@ Properties props = new Properties(); props.setProperty("soap_host", "my.host.com"); //$NON-NLS-1$ //$NON-NLS-2$ props.setProperty("soap_port", "12345"); //$NON-NLS-1$ //$NON-NLS-2$ - CommandContext context = new CommandContext("0", "test", "user", null, "myvdb", 1, props, false); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ + CommandContext context = new CommandContext("0", "test", "user", null, "myvdb", 1, props, DEBUG); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ context.setProcessorBatchSize(BufferManager.DEFAULT_PROCESSOR_BATCH_SIZE); context.setConnectorBatchSize(BufferManager.DEFAULT_CONNECTOR_BATCH_SIZE); context.setBufferManager(BufferManagerFactory.getStandaloneBufferManager()); @@ -7570,5 +7570,24 @@ helpProcess(plan, dataManager, new List[] {Arrays.asList(1, 1)}); } + @Test public void testOrderByExpression() throws Exception { + String sql = "SELECT pm1.g1.e2 as y FROM pm1.g1 ORDER BY e3 || e1"; + + List[] expected = new List[] { + Arrays.asList(0), + Arrays.asList(1), + Arrays.asList(3), + Arrays.asList(1), + Arrays.asList(2), + Arrays.asList(0), + }; + + FakeDataManager dataManager = new FakeDataManager(); + sampleData1(dataManager); + ProcessorPlan plan = helpGetPlan(sql, RealMetadataFactory.example1Cached(), TestOptimizer.getGenericFinder()); + + helpProcess(plan, dataManager, expected); + } + private static final boolean DEBUG = false; } Modified: trunk/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/processor/TestVirtualDepJoin.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -364,7 +364,7 @@ List<String> expectedQueries = new ArrayList<String>(6); for (int i = 0; i < 3; i++) { expectedQueries.add("SELECT g_0.id AS c_0, g_0.first AS c_1, g_0.last AS c_2 FROM CustomerMaster.Customers AS g_0 WHERE g_0.first = 'Miles' ORDER BY c_0"); //$NON-NLS-1$ - expectedQueries.add("SELECT g_0.id AS c_0, g_0.amount AS c_1 FROM Europe.CustAccts AS g_0 WHERE g_0.id = 100 ORDER BY c_0"); //$NON-NLS-1$ + expectedQueries.add("SELECT g_0.id, g_0.amount FROM Europe.CustAccts AS g_0 WHERE g_0.id = 100"); //$NON-NLS-1$ } assertEquals(expectedQueries, dataManager.getQueries()); Modified: trunk/engine/src/test/java/org/teiid/query/sql/symbol/TestSearchedCaseExpression.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/sql/symbol/TestSearchedCaseExpression.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/sql/symbol/TestSearchedCaseExpression.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -25,16 +25,11 @@ import java.util.ArrayList; import java.util.List; +import junit.framework.TestCase; + import org.teiid.core.types.DataTypeManager; import org.teiid.query.sql.lang.CompareCriteria; -import org.teiid.query.sql.symbol.Constant; -import org.teiid.query.sql.symbol.ElementSymbol; -import org.teiid.query.sql.symbol.Expression; -import org.teiid.query.sql.symbol.SearchedCaseExpression; - -import junit.framework.TestCase; - public class TestSearchedCaseExpression extends TestCase { /** Modified: trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestExpressionMappingVisitor.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestExpressionMappingVisitor.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/sql/visitor/TestExpressionMappingVisitor.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -45,7 +45,6 @@ import org.teiid.query.sql.symbol.SearchedCaseExpression; import org.teiid.query.sql.symbol.TestCaseExpression; import org.teiid.query.sql.symbol.TestSearchedCaseExpression; -import org.teiid.query.sql.visitor.ExpressionMappingVisitor; import org.teiid.translator.SourceSystemFunctions; @@ -196,13 +195,13 @@ * a function. */ @Test public void testRecursionDetection() { - ElementSymbol e1 = new ElementSymbol("e1"); //$NON-NLS-1$ + ElementSymbol e1 = new ElementSymbol("g1.e1"); //$NON-NLS-1$ AggregateSymbol a1 = new AggregateSymbol("x", NonReserved.SUM, false, e1); //$NON-NLS-1$ Function f = new Function(SourceSystemFunctions.ADD_OP, new Expression[] {a1, a1}); HashMap<AggregateSymbol, AggregateSymbol> map = new HashMap<AggregateSymbol, AggregateSymbol>(); map.put(a1, new AggregateSymbol("x", NonReserved.SUM, false, a1)); //$NON-NLS-1$ ExpressionMappingVisitor.mapExpressions(f, map); - assertEquals("(SUM(SUM(e1)) + SUM(SUM(e1)))", f.toString()); //$NON-NLS-1$ + assertEquals("(SUM(SUM(g1.e1)) + SUM(SUM(g1.e1)))", f.toString()); //$NON-NLS-1$ } } Modified: trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java =================================================================== --- trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/engine/src/test/java/org/teiid/query/validator/TestValidator.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -1839,5 +1839,25 @@ models.add("MultiModel"); helpValidate("exec MultiModel.proc('a', (select 1))", new String[] {"MultiModel.proc.source_name"}, new MultiSourceMetadataWrapper(RealMetadataFactory.exampleMultiBinding(), models)); //$NON-NLS-1$ } + + @Test public void testFailAggregateInGroupBy() { + helpValidate("SELECT max(e1) FROM pm1.g1 GROUP BY count(e2)", new String[] {"COUNT(e2)"}, RealMetadataFactory.example1Cached()); + } + + @Test public void testFailAggregateInWhere() { + helpValidate("SELECT e1 FROM pm1.g1 where count(e2) = 1", new String[] {"COUNT(e2)"}, RealMetadataFactory.example1Cached()); + } + @Test public void testFailAggregateInFrom() { + helpValidate("SELECT g2.e1 FROM pm1.g1 inner join pm1.g2 on (avg(g1.e2) = g2.e2)", new String[] {"AVG(g1.e2)"}, RealMetadataFactory.example1Cached()); + } + + @Test public void testFailAggregateFilterSubquery() { + helpValidate("SELECT min(g1.e1) filter (where (select 1) = 1) from pm1.g1", new String[] {"(SELECT 1) = 1"}, RealMetadataFactory.example1Cached()); + } + + @Test public void testNestedAgg() { + helpValidate("SELECT min(g1.e1) filter (where max(e2) = 1) from pm1.g1", new String[] {"MAX(e2)"}, RealMetadataFactory.example1Cached()); + } + } Modified: trunk/test-integration/common/src/test/java/org/teiid/dqp/internal/process/TestTPCR.java =================================================================== --- trunk/test-integration/common/src/test/java/org/teiid/dqp/internal/process/TestTPCR.java 2011-07-19 13:41:13 UTC (rev 3316) +++ trunk/test-integration/common/src/test/java/org/teiid/dqp/internal/process/TestTPCR.java 2011-07-20 03:27:02 UTC (rev 3317) @@ -191,7 +191,7 @@ ProcessorPlan plan = TestOptimizer.helpPlan("SELECT custsale.cntrycode, COUNT(*) AS numcust, SUM(c_acctbal) AS totacctbal FROM (SELECT left(C_PHONE, 2) AS cntrycode, CUSTOMER.C_ACCTBAL FROM CUSTOMER WHERE (left(C_PHONE, 2) IN ('13','31','23','29','30','18','17')) AND (CUSTOMER.C_ACCTBAL > (SELECT AVG(CUSTOMER.C_ACCTBAL) FROM CUSTOMER WHERE (CUSTOMER.C_ACCTBAL > 0.0) AND (left(C_PHONE, 2) IN ('13','31','23','29','30','18','17')))) AND (NOT (EXISTS (SELECT * FROM ORDERS WHERE O_CUSTKEY = C_CUSTKEY)))) AS custsale GROUP BY custsale.cntrycode ORDER BY custsale.cntrycode", //$NON-NLS-1$ METADATA, null, finder, - new String[] {"SELECT v_0.c_0, COUNT(*) AS c_1, SUM(v_0.c_1) AS c_2 FROM (SELECT left(g_0.C_PHONE, 2) AS c_0, g_0.C_ACCTBAL AS c_1 FROM TPCR_Oracle_9i.CUSTOMER AS g_0 WHERE (left(g_0.C_PHONE, 2) IN ('13', '31', '23', '29', '30', '18', '17')) AND (g_0.C_ACCTBAL > (SELECT AVG(g_1.C_ACCTBAL) FROM TPCR_Oracle_9i.CUSTOMER AS g_1 WHERE (g_1.C_ACCTBAL > 0E-15) AND (left(g_1.C_PHONE, 2) IN ('13', '31', '23', '29', '30', '18', '17')))) AND (NOT EXISTS (SELECT 1 FROM TPCR_Oracle_9i.ORDERS AS g_2 WHERE g_2.O_CUSTKEY = g_0.C_CUSTKEY))) AS v_0 GROUP BY v_0.c_0 ORDER BY c_0 NULLS FIRST"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ + new String[] {"SELECT left(g_0.C_PHONE, 2) AS c_0, COUNT(*) AS c_1, SUM(g_0.C_ACCTBAL) AS c_2 FROM TPCR_Oracle_9i.CUSTOMER AS g_0 WHERE (left(g_0.C_PHONE, 2) IN ('13', '31', '23', '29', '30', '18', '17')) AND (g_0.C_ACCTBAL > (SELECT AVG(g_1.C_ACCTBAL) FROM TPCR_Oracle_9i.CUSTOMER AS g_1 WHERE (g_1.C_ACCTBAL > 0E-15) AND (left(g_1.C_PHONE, 2) IN ('13', '31', '23', '29', '30', '18', '17')))) AND (NOT EXISTS (SELECT 1 FROM TPCR_Oracle_9i.ORDERS AS g_2 WHERE g_2.O_CUSTKEY = g_0.C_CUSTKEY)) GROUP BY left(g_0.C_PHONE, 2) ORDER BY c_0 NULLS FIRST"}, ComparisonMode.EXACT_COMMAND_STRING); //$NON-NLS-1$ TestOptimizer.checkNodeTypes(plan, TestOptimizer.FULL_PUSHDOWN); } [View Less]

13 years, 9 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-commits July 2011