teiid SVN: r2502 - in branches/7.1.x: documentation/caching-guide/src/main/docbook/en-US/content and 2 other directories.
by teiid-commits＠lists.jboss.org
Author: shawkins
Date: 2010-08-30 11:39:33 -0400 (Mon, 30 Aug 2010)
New Revision: 2502
Modified:
branches/7.1.x/build/kits/jboss-container/teiid-releasenotes.html
branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml
branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml
branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
Log:
TEIID-1210 TEIID-1220 adding release note entries, updating docs, and adding more logging.
Modified: branches/7.1.x/build/kits/jboss-container/teiid-releasenotes.html
===================================================================
--- branches/7.1.x/build/kits/jboss-container/teiid-releasenotes.html 2010-08-30 03:21:37 UTC (rev 2501)
+++ branches/7.1.x/build/kits/jboss-container/teiid-releasenotes.html 2010-08-30 15:39:33 UTC (rev 2502)
@@ -29,12 +29,18 @@
<LI><B>Procedure Result Caching</B> - virtual procedure definitions may use a cache hint to cache results in the result set cache.
<LI><B>Improved Plan Caching</B> - plans used by internal materialization and stored procedure plans will be automatically cached in the prepared plan cache. Improvements were also made to reduce the memory footprint of the plans.
<LI><B>Refined Load Balancing and Fail Over</B> - clients can use use the statement "SET NEWINSTANCE TRUE" to allow their connection to select a new server instance. See the Client Developer's Guide for more information.
+ <LI><B>Simplified Role Usage</B> - a role can now be assigned to any authenticated user via the any-authenticated attribute on the data-role element.
+ <LI><B>Materialized View Performance</B> - materialized view tables will now automatically create and use non-unique secondary indexes for unique constraints and indexes defined on the view.
</UL>
<h2><a name="Compatibility">Compatibility Issues</a></h2>
<ul>
<li>Support for named parameter syntax using param=value has been deprecated, since it is ambiguous with a comparison predicate boolean value expression. param<b>=></b>value should be used instead.
</ul>
+<h4>from 7.1</h4>
+<ul>
+ <li>Model visibility no longer restricts access to tables and procedures. Setting visible to false will only hide entries from system tables. Data roles should be used to restrict data access.
+</ul>
<h4>from 7.0</h4>
<ul>
<li>The term data policy was replaced with data role. The AdminAPI methods for adding role mappings have changed from addRoleToDataPolicy and removeRoleFromDataPolicy to addDataRoleMapping and removeDataRoleMapping respectively.
Modified: branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml
===================================================================
--- branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml 2010-08-30 03:21:37 UTC (rev 2501)
+++ branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml 2010-08-30 15:39:33 UTC (rev 2502)
@@ -60,7 +60,7 @@
<para>More control of the cache hint.</para>
</listitem>
<listitem>
- <para>The ability to use option nocache.</para>
+ <para>The ability to use <link linkend="nocache">OPTION NOCACHE</link>.</para>
</listitem>
<listitem>
<para>Usage of a materialized view lookup as an uncorrelated subquery is no different than the use of the lookup function.</para>
Modified: branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml
===================================================================
--- branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml 2010-08-30 03:21:37 UTC (rev 2501)
+++ branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml 2010-08-30 15:39:33 UTC (rev 2502)
@@ -162,9 +162,21 @@
</para>
</section>
</section>
- <section>
+ <section id="internal-index">
<title>Secondary Indexes</title>
- <para>Function based indexes, Covering indexes, converting unique to index</para>
+ <para>Internal materialized view tables will automatically create non-unique indexes for each unique constraint and index defined on the materialized view.
+ These indexes are created as non-unique even for unique constraints since the materialized table is not intended as an enforcement point for data integrity
+ and when updatable the table may not be consistent with underlying values and thus unable to satisfy constraints. The primary key (if it exists) of the view will automatically be part of the covered columns for the index.</para>
+ <para>The secondary indexes are always created as trees - bitmap or hash indexes are not supported. Teiid's metadata for indexes is currently limited.
+ We are not currently able to capture additional information, such as specifying the evluated expressions, sort direction, additional columns to cover, etc. You may workaround some of these limitations though.
+ <itemizedlist>
+ <listitem><para>If a function based index is needed, consider adding another column to the view that projects the funciton expression, then place an index on that new column.
+ Queries to the view will need to be modified as appropiate though to make use of the new column/index.</para></listitem>
+ <listitem><para>If additional covered columns are needed, they may simply be added to the index columns. This however is only applicable to comparable types.
+ Adding additional columns will increase the amount of space used by the index, but may allow
+ its usage to result in higher performance when only the covered columns are used and the main table is not consulted.</para></listitem>
+ </itemizedlist>
+ </para>
</section>
</section>
</chapter>
Modified: branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml
===================================================================
--- branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml 2010-08-30 03:21:37 UTC (rev 2501)
+++ branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml 2010-08-30 15:39:33 UTC (rev 2502)
@@ -6,11 +6,18 @@
<chapter id="dataroles">
<title>Data Roles</title>
<para>Data roles, also called entitlements, are sets of permissions defined
- per VDB that dictate data access (create, read, update, delete). The use of data roles can be disabled system wide with the property in
- &jboss-beans; file in bean configuration section of <code>RuntimeEngineDeployer</code> with property <code>useDataRoles</code>.</para>
+ per VDB that dictate data access (create, read, update, delete). Data roles use a fine-grained permission system that Teiid will enforce
+ at runtime and provide audit log entries for access violations (see that Admin and Developers Guide for more on Audit Logging).</para>
- <para>If data roles is enabled and data roles are defined in a VDB, then access permissions will be enforced by the Teiid Server.</para>
+ <para>Prior to applying data roles, you should consider restricting source system access through the fundamental design of your VDB.
+ Foremost, Teiid can only access source entries that are represented in imported metadata. You should narrow imported metadata to only what
+ is necessary for use by your VDB. When using Teiid Designer, you may then go further and modify the imported metadata at a granular
+ level to remove specific columns, mark tables as non-updatable, etc.</para>
+ <para>If data roles is enabled and data roles are defined in a VDB, then access permissions will be enforced by the Teiid Server.
+ The use of data roles may be disabled system wide via the
+ &jboss-beans; file, by setting the property <code>useDataRoles</code> to false in the configuration section of the <code>RuntimeEngineDeployer</code>.</para>
+
<warning><para>Unlike previous versions of Teiid data roles will only be checked if present in a VDB.
A VDB deployed without data roles is open for use by any authenticated user.</para></warning>
@@ -81,14 +88,15 @@
<section>
<title>XML Definition</title>
- <para>Data roles are defined inside the <code>vdb.xml</code> file (inside the .vdb Zip archive under META-INF/vdb.xml) if you used Designer.
- This example will show a sample "vdb.xml" file with few simple data rules.</para>
+ <para>Data roles are defined inside the <code>vdb.xml</code> file (inside the .vdb Zip archive under META-INF/vdb.xml) if you used Designer.
+ The "vdb.xml" file is checked against the schema file <code>vdb-deployer.xsd</code>, which can be found in the kit under teiid-docs/schema.
+ This example will show a sample "vdb.xml" file with few simple data roles.</para>
<para>For example, if a VDB defines a table "TableA" in schema "modelName" with columns (column1, column2) - note that the column types do not matter. And we wish to define three roles "RoleA", "RoleB", "RoleC" with following permissions:
<orderedlist>
- <listitem><para>RoleA has privileges to read, write access to TableA, but can not delete.</para></listitem>
- <listitem><para>RoleB has no privileges that allow access to TableA</para></listitem>
- <listitem><para>RoleC has privileges that only allow read access to TableA.column1</para></listitem>
+ <listitem><para>RoleA has permissions to read, write access to TableA, but can not delete.</para></listitem>
+ <listitem><para>RoleB has no permissions that allow access to TableA</para></listitem>
+ <listitem><para>RoleC has permissions that only allow read access to TableA.column1</para></listitem>
</orderedlist>
</para>
<example><title>vdb.xml defining RoleA, RoleB, and RoleC</title>
@@ -109,20 +117,6 @@
<allow-update>true</allow-update>
</permission>
- <permission>
- <resource-name>modelName.TableA.colum1</resource-name>
- <allow-create>true</allow-create>
- <allow-read>true</allow-read>
- <allow-update>true</allow-update>
- </permission>
-
- <permission>
- <resource-name>modelName.TableA.column2</resource-name>
- <allow-create>true</allow-create>
- <allow-read>true</allow-read>
- <allow-update>true</allow-update>
- </permission>
-
<mapped-role-name>role1</mapped-role-name>
</data-role>
@@ -136,8 +130,8 @@
</permission>
<permission>
- <resource-name>modelName.TableA.colum1</resource-name>
- <allow-read>true</allow-read>
+ <resource-name>modelName.TableA.colum2</resource-name>
+ <allow-read>false</allow-read>
</permission>
<mapped-role-name>role2</mapped-role-name>
@@ -145,14 +139,17 @@
</vdb>]]></programlisting>
</example>
<para>The above XML defined two data roles, "RoleA" which allows everything except delete on the table, "RoleC" that
- allows only read operation on the table. Since Teiid uses deny by default, there is no explicit data-role entry needed for "RoleB". The "mapped-role-name" defines the JAAS "role" to whom these policies are applicable.</para>
+ allows only read operation on the table. Since Teiid uses deny by default, there is no explicit data-role entry needed for "RoleB". Note that explicit column permissions are
+ not needed for RoleA, since the parent resource path, modelName.TableA, permissions still apply. RoleC however must explicitly disallow read to column2.</para>
- <para>For assigning roles to your users in the JBoss AS,
- check out the instructions for the selected Login Module. Check the "Admin Guide" for configuring Login Modules.</para>
+ <para>The "mapped-role-name" defines the container JAAS roles that are assigned the data role. For assigning roles to your users in the JBoss AS,
+ check out the instructions for the selected Login Module. Check the "Admin Guide" for configuring Login Modules. You may also choose to allow any
+ authenticated user to have a data role by setting the any-authenticated attribute value to true on data-role element.</para>
- <para>The "vdb.xml" file is checked against the schema file <code>vdb-deployer.xsd</code>, check the documents sections of the Teiid kit
- to find a copy of the schema file.</para>
-
</section>
-
+ <section>
+ <title>System Functions</title>
+ <para>The <code>hasRole</code> system function will return true if the current user has the given data role.
+ The <code>hasRole</code> function can be used in procedure or view definitions to allow for a more dynamic application of security - which allows for things such as value masking or row level security.</para>
+ </section>
</chapter>
\ No newline at end of file
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java 2010-08-30 03:21:37 UTC (rev 2501)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java 2010-08-30 15:39:33 UTC (rev 2502)
@@ -32,6 +32,8 @@
import org.teiid.common.buffer.TupleBrowser;
import org.teiid.common.buffer.TupleSource;
import org.teiid.core.TeiidComponentException;
+import org.teiid.logging.LogConstants;
+import org.teiid.logging.LogManager;
import org.teiid.query.processor.CollectionTupleSource;
import org.teiid.query.processor.relational.RelationalNode;
import org.teiid.query.sql.lang.CompareCriteria;
@@ -67,23 +69,25 @@
covering = true;
}
if (table.getPkLength() > 0) {
- processCriteria(condition);
+ processCriteria(condition, primary);
if (orderBy != null && (covering || this.table.getColumnMap().keySet().containsAll(orderBy.getSortKeys()))) {
ordering = useIndexForOrderBy(orderBy);
}
}
}
- private void processCriteria(Criteria condition) {
+ private void processCriteria(Criteria condition, boolean primary) {
List<Criteria> crits = Criteria.separateCriteriaByAnd(condition);
- for (Iterator<Criteria> critIter = crits.iterator(); critIter.hasNext();) {
- Criteria criteria = critIter.next();
- if (table.getColumnMap().keySet().containsAll(ElementCollectorVisitor.getElements(criteria, false))) {
- coveredCriteria.add(criteria);
- } else {
- covering = false;
- nonCoveredCriteria.add(criteria);
- critIter.remove();
+ if (!primary) {
+ for (Iterator<Criteria> critIter = crits.iterator(); critIter.hasNext();) {
+ Criteria criteria = critIter.next();
+ if (table.getColumnMap().keySet().containsAll(ElementCollectorVisitor.getElements(criteria, false))) {
+ coveredCriteria.add(criteria);
+ } else {
+ covering = false;
+ nonCoveredCriteria.add(criteria);
+ critIter.remove();
+ }
}
}
for (int i = 0; i < table.getPkLength(); i++) {
@@ -244,11 +248,17 @@
}
TupleBrowser createTupleBrowser() throws TeiidComponentException {
- boolean direction = ordering == null ? OrderBy.ASC : ordering;
+ boolean direction = OrderBy.ASC;
+ if (ordering != null) {
+ LogManager.logDetail(LogConstants.CTX_DQP, "Using index for ordering"); //$NON-NLS-1$
+ direction = ordering;
+ }
if (valueTs != null) {
+ LogManager.logDetail(LogConstants.CTX_DQP, "Using index value set"); //$NON-NLS-1$
return new TupleBrowser(this.table.getTree(), valueTs, direction);
}
if (!valueSet.isEmpty()) {
+ LogManager.logDetail(LogConstants.CTX_DQP, "Using index value set"); //$NON-NLS-1$
CollectionTupleSource cts = null;
if (direction == OrderBy.ASC) {
cts = new CollectionTupleSource(valueSet.iterator());
@@ -271,6 +281,9 @@
}
return new TupleBrowser(this.table.getTree(), cts, direction);
}
+ if (lower != null || upper != null) {
+ LogManager.logDetail(LogConstants.CTX_DQP, "Using index for range query", lower, upper); //$NON-NLS-1$
+ }
return new TupleBrowser(this.table.getTree(), lower, upper, direction);
}
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-30 03:21:37 UTC (rev 2501)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-30 15:39:33 UTC (rev 2502)
@@ -315,6 +315,7 @@
IndexInfo primary = new IndexInfo(this, projectedCols, condition, orderBy, true);
IndexInfo ii = primary;
if (indexTables != null && (condition != null || orderBy != null) && ii.valueSet.size() != 1) {
+ LogManager.logDetail(LogConstants.CTX_DQP, "Considering indexes on table", this, "for query", projectedCols, condition, orderBy); //$NON-NLS-1$ //$NON-NLS-2$
int rowCost = this.tree.getRowCount();
int bestCost = estimateCost(orderBy, ii, rowCost);
for (TempTable table : this.indexTables.values()) {
@@ -325,6 +326,7 @@
bestCost = cost;
}
}
+ LogManager.logDetail(LogConstants.CTX_DQP, "Choose index", ii.table, "covering:", ii.coveredCriteria,"ordering:", ii.ordering); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
if (ii.covering) {
return ii.table.createTupleSource(projectedCols, condition, orderBy, ii);
}
@@ -345,7 +347,6 @@
Criteria.combineCriteria(ii.coveredCriteria), pkOrderBy, ii);
return createTupleSource(projectedCols, Criteria.combineCriteria(ii.nonCoveredCriteria), orderBy, primary);
}
-
return createTupleSource(projectedCols, condition, orderBy, ii);
}
13 years, 8 months
- 1
- 0
teiid SVN: r2501 - in branches/7.1.x: engine/src/main/java/org/teiid/query/metadata and 3 other directories.
by teiid-commits＠lists.jboss.org
Author: shawkins
Date: 2010-08-29 23:21:37 -0400 (Sun, 29 Aug 2010)
New Revision: 2501
Modified:
branches/7.1.x/api/src/main/java/org/teiid/metadata/KeyRecord.java
branches/7.1.x/api/src/main/java/org/teiid/metadata/MetadataFactory.java
branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java
branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java
Log:
TEIID-1210 adding secondary index support to internal materialized views
Modified: branches/7.1.x/api/src/main/java/org/teiid/metadata/KeyRecord.java
===================================================================
--- branches/7.1.x/api/src/main/java/org/teiid/metadata/KeyRecord.java 2010-08-29 14:03:30 UTC (rev 2500)
+++ branches/7.1.x/api/src/main/java/org/teiid/metadata/KeyRecord.java 2010-08-30 03:21:37 UTC (rev 2501)
@@ -30,6 +30,7 @@
Primary,
Foreign,
Unique, //constraint
+ @Deprecated
NonUnique,
AccessPattern,
Index,
@@ -42,6 +43,9 @@
}
public Type getType() {
+ if (type == Type.NonUnique) {
+ type = Type.Index;
+ }
return type;
}
Modified: branches/7.1.x/api/src/main/java/org/teiid/metadata/MetadataFactory.java
===================================================================
--- branches/7.1.x/api/src/main/java/org/teiid/metadata/MetadataFactory.java 2010-08-29 14:03:30 UTC (rev 2500)
+++ branches/7.1.x/api/src/main/java/org/teiid/metadata/MetadataFactory.java 2010-08-30 03:21:37 UTC (rev 2501)
@@ -170,7 +170,7 @@
* @throws TranslatorException
*/
public KeyRecord addIndex(String name, boolean nonUnique, List<String> columnNames, Table table) throws TranslatorException {
- KeyRecord index = new KeyRecord(nonUnique?KeyRecord.Type.NonUnique:KeyRecord.Type.Index);
+ KeyRecord index = new KeyRecord(nonUnique?KeyRecord.Type.Index:KeyRecord.Type.Unique);
index.setParent(table);
index.setColumns(new ArrayList<Column>(columnNames.size()));
index.setName(name);
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java 2010-08-29 14:03:30 UTC (rev 2500)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TransformationMetadata.java 2010-08-30 03:21:37 UTC (rev 2501)
@@ -614,7 +614,7 @@
result.add(tableRecordImpl.getPrimaryKey());
}
for (KeyRecord key : tableRecordImpl.getIndexes()) {
- if (key.getType() == KeyRecord.Type.Index) {
+ if (key.getType() == KeyRecord.Type.Unique) {
result.add(key);
}
}
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java 2010-08-29 14:03:30 UTC (rev 2500)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java 2010-08-30 03:21:37 UTC (rev 2501)
@@ -24,6 +24,7 @@
import java.util.ArrayList;
import java.util.Iterator;
+import java.util.LinkedList;
import java.util.List;
import java.util.ListIterator;
import java.util.TreeSet;
@@ -43,6 +44,7 @@
import org.teiid.query.sql.symbol.Constant;
import org.teiid.query.sql.symbol.ElementSymbol;
import org.teiid.query.sql.symbol.SingleElementSymbol;
+import org.teiid.query.sql.visitor.ElementCollectorVisitor;
/**
* Accumulates information about index usage.
@@ -56,6 +58,8 @@
Boolean ordering;
boolean covering;
TupleSource valueTs;
+ List<Criteria> nonCoveredCriteria = new LinkedList<Criteria>();
+ List<Criteria> coveredCriteria = new LinkedList<Criteria>();
public IndexInfo(TempTable table, final List<? extends SingleElementSymbol> projectedCols, final Criteria condition, OrderBy orderBy, boolean primary) {
this.table = table;
@@ -72,6 +76,16 @@
private void processCriteria(Criteria condition) {
List<Criteria> crits = Criteria.separateCriteriaByAnd(condition);
+ for (Iterator<Criteria> critIter = crits.iterator(); critIter.hasNext();) {
+ Criteria criteria = critIter.next();
+ if (table.getColumnMap().keySet().containsAll(ElementCollectorVisitor.getElements(criteria, false))) {
+ coveredCriteria.add(criteria);
+ } else {
+ covering = false;
+ nonCoveredCriteria.add(criteria);
+ critIter.remove();
+ }
+ }
for (int i = 0; i < table.getPkLength(); i++) {
ElementSymbol keyColumn = table.getColumns().get(i);
for (Iterator<Criteria> critIter = crits.iterator(); critIter.hasNext();) {
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-29 14:03:30 UTC (rev 2500)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-30 03:21:37 UTC (rev 2501)
@@ -70,8 +70,6 @@
*/
class TempTable {
- private static final double LN_2 = Math.log(2);
-
private final class InsertUpdateProcessor extends UpdateProcessor {
private boolean addRowId;
@@ -131,7 +129,9 @@
this.condition = condition;
this.project = shouldProject();
this.reserved = reserveBuffers();
- lock.readLock().lock();
+ if (updatable) {
+ lock.readLock().lock();
+ }
}
@Override
@@ -144,9 +144,6 @@
reserved = 0;
return null;
}
- if (rowId != null) {
- next = next.subList(1, next.size());
- }
if (condition != null && !eval.evaluate(condition, next)) {
continue;
}
@@ -159,7 +156,9 @@
@Override
public void closeSource() {
- lock.readLock().unlock();
+ if (updatable) {
+ lock.readLock().unlock();
+ }
bm.releaseBuffers(reserved);
reserved = 0;
browser.closeSource();
@@ -269,7 +268,6 @@
TempTable(TempMetadataID tid, BufferManager bm, List<ElementSymbol> columns, int primaryKeyLength, String sessionID) {
this.tid = tid;
this.bm = bm;
- this.columnMap = RelationalNode.createLookupMap(columns);
if (primaryKeyLength == 0) {
ElementSymbol id = new ElementSymbol("rowId"); //$NON-NLS-1$
id.setType(DataTypeManager.DefaultDataClasses.INTEGER);
@@ -279,6 +277,7 @@
} else {
tree = bm.createSTree(columns, sessionID, primaryKeyLength);
}
+ this.columnMap = RelationalNode.createLookupMap(columns);
this.columns = columns;
this.sessionID = sessionID;
this.keyBatchSize = bm.getSchemaSize(columns);
@@ -296,8 +295,9 @@
allColumns.add(elementSymbol);
}
}
- TempTable indexTable = new TempTable(new TempMetadataID("idx", Collections.EMPTY_LIST), this.bm, allColumns, indexColumns.size(), this.sessionID); //$NON-NLS-1$
+ TempTable indexTable = new TempTable(new TempMetadataID("idx", Collections.EMPTY_LIST), this.bm, allColumns, allColumns.size(), this.sessionID); //$NON-NLS-1$
indexTable.setPreferMemory(this.tree.isPreferMemory());
+ indexTable.lock = this.lock;
if (indexTables == null) {
indexTables = new LinkedHashMap<List<ElementSymbol>, TempTable>();
indexTables.put(indexColumns, indexTable);
@@ -328,20 +328,22 @@
if (ii.covering) {
return ii.table.createTupleSource(projectedCols, condition, orderBy, ii);
}
- List<ElementSymbol> pkColumns = this.columns.subList(0, this.getPkLength());
+ List<ElementSymbol> pkColumns = this.columns.subList(0, this.tree.getKeyLength());
if (ii.ordering != null) {
//use order and join
- primary.valueTs = ii.table.createTupleSource(pkColumns, condition, orderBy, ii);
+ primary.valueTs = ii.table.createTupleSource(pkColumns,
+ Criteria.combineCriteria(ii.coveredCriteria), orderBy, ii);
primary.ordering = null;
- return createTupleSource(projectedCols, condition, null, primary);
+ return createTupleSource(projectedCols, Criteria.combineCriteria(ii.nonCoveredCriteria), null, primary);
}
//order by pk to localize lookup costs, then join
OrderBy pkOrderBy = new OrderBy();
for (ElementSymbol elementSymbol : pkColumns) {
pkOrderBy.addVariable(elementSymbol);
}
- primary.valueTs = ii.table.createTupleSource(pkColumns, condition, pkOrderBy, ii);
- return createTupleSource(projectedCols, condition, orderBy, primary);
+ primary.valueTs = ii.table.createTupleSource(pkColumns,
+ Criteria.combineCriteria(ii.coveredCriteria), pkOrderBy, ii);
+ return createTupleSource(projectedCols, Criteria.combineCriteria(ii.nonCoveredCriteria), orderBy, primary);
}
return createTupleSource(projectedCols, condition, orderBy, ii);
@@ -383,20 +385,20 @@
/**
* TODO: this could easily use statistics - the tree level 1 would be an ideal place
* to compute them, since it minimizes page loads, and is a random sample.
- * @return
+ * TODO: this should also factor in the block size
*/
private int estimateCost(OrderBy orderBy, IndexInfo ii, int rowCost) {
if (ii.valueSet.size() != 0) {
int length = ii.valueSet.get(0).size();
- rowCost = Math.min(rowCost, ii.valueSet.size() * 1 << (ii.table.getPkLength() - length));
+ rowCost = ii.valueSet.size() * (ii.table.getPkLength() - length + 1);
} else if (ii.upper != null) {
rowCost /= 3;
} else if (ii.lower != null) {
rowCost /= 3;
}
int cost = Math.max(1, rowCost);
- if (!ii.covering || (orderBy != null && ii.ordering == null)) {
- cost = (int)(cost * Math.log(cost)/LN_2);
+ if (cost > 1 && (!ii.covering || (orderBy != null && ii.ordering == null))) {
+ cost *= (32 - Integer.numberOfLeadingZeros(cost - 1));
}
return cost;
}
@@ -542,7 +544,7 @@
return CollectionTupleSource.createUpdateCountTupleSource(updateCount);
}
- private void insertTuple(List<Object> list, boolean ordered) throws TeiidComponentException, TeiidProcessingException {
+ private void insertTuple(List<?> list, boolean ordered) throws TeiidComponentException, TeiidProcessingException {
if (tree.insert(list, ordered?InsertMode.ORDERED:InsertMode.NEW) != null) {
throw new TeiidProcessingException(QueryPlugin.Util.getString("TempTable.duplicate_key")); //$NON-NLS-1$
}
@@ -563,17 +565,19 @@
return null;
}
if (indexTables != null) {
- //remove from each index table
- /*for (TempTable index : this.indexTables.values()) {
- index.tree
- }*/
+ for (TempTable index : this.indexTables.values()) {
+ tuple = RelationalNode.projectTuple(RelationalNode.getProjectionIndexes(index.getColumnMap(), index.columns), result);
+ index.tree.remove(tuple);
+ }
}
return result;
}
List<?> result = tree.insert(tuple, InsertMode.UPDATE);
if (indexTables != null) {
- //update each index table
-
+ for (TempTable index : this.indexTables.values()) {
+ tuple = RelationalNode.projectTuple(RelationalNode.getProjectionIndexes(index.getColumnMap(), index.columns), tuple);
+ index.tree.insert(tuple, InsertMode.UPDATE);
+ }
}
return result;
} finally {
Modified: branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java
===================================================================
--- branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java 2010-08-29 14:03:30 UTC (rev 2500)
+++ branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java 2010-08-30 03:21:37 UTC (rev 2501)
@@ -123,6 +123,12 @@
@Test public void testNonCoveringSecondaryIndex() throws Exception {
execute("SELECT * from vgroup5 where y in ('zne', 'zwo') order by y desc", Arrays.asList("two", "zwo", 1), Arrays.asList("one", "zne", 1));
execute("SELECT * from vgroup5 where y is null", Arrays.asList((String)null, (String)null, 1));
+ execute("SELECT * from vgroup5 where y is null and z = 2");
}
+
+ @Test public void testNonCoveringSecondaryIndexWithoutPrimaryKey() throws Exception {
+ execute("SELECT * from vgroup6 where y in ('zne', 'zwo') order by y desc", Arrays.asList("two", "zwo"), Arrays.asList("one", "zne"));
+ execute("SELECT * from vgroup6 where y is null", Arrays.asList((String)null, (String)null));
+ }
}
Modified: branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java
===================================================================
--- branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java 2010-08-29 14:03:30 UTC (rev 2500)
+++ branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java 2010-08-30 03:21:37 UTC (rev 2501)
@@ -372,6 +372,16 @@
createKey(KeyRecord.Type.Primary, "pk", vGroup5, vElements5.subList(0, 1));
createKey(KeyRecord.Type.Index, "idx", vGroup5, vElements5.subList(1, 2));
+ //no pk
+ QueryNode vTrans6 = new QueryNode("VGroup6", "SELECT x, 'z' || substring(x, 2) as y FROM matsrc"); //$NON-NLS-1$ //$NON-NLS-2$
+ Table vGroup6 = createVirtualGroup("VGroup6", virtModel, vTrans6); //$NON-NLS-1$
+ vGroup6.setMaterialized(true);
+ List<Column> vElements6 = createElements(vGroup6,
+ new String[] { "x", "y" }, //$NON-NLS-1$
+ new String[] { DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING});
+
+ createKey(KeyRecord.Type.Index, "idx", vGroup6, vElements6.subList(1, 2));
+
Schema sp = createVirtualModel("sp", metadataStore); //$NON-NLS-1$
ColumnSet<Procedure> rs = createResultSet("sp1.vsprs1", new String[] { "StringKey" }, new String[] { DataTypeManager.DefaultDataTypes.STRING }); //$NON-NLS-1$ //$NON-NLS-2$
ProcedureParameter param = createParameter("param1", ParameterInfo.IN, DataTypeManager.DefaultDataTypes.STRING); //$NON-NLS-1$
13 years, 8 months
- 1
- 0
teiid SVN: r2500 - branches/7.1.x/connectors/connector-ldap/src/main/rar/META-INF.
by teiid-commits＠lists.jboss.org
Author: rareddy
Date: 2010-08-29 10:03:30 -0400 (Sun, 29 Aug 2010)
New Revision: 2500
Modified:
branches/7.1.x/connectors/connector-ldap/src/main/rar/META-INF/ra.xml
Log:
TEIID-1232: the class definition in the ra.xml was defined wrong. This was defined for previous package structure.
Modified: branches/7.1.x/connectors/connector-ldap/src/main/rar/META-INF/ra.xml
===================================================================
--- branches/7.1.x/connectors/connector-ldap/src/main/rar/META-INF/ra.xml 2010-08-27 21:20:51 UTC (rev 2499)
+++ branches/7.1.x/connectors/connector-ldap/src/main/rar/META-INF/ra.xml 2010-08-29 14:03:30 UTC (rev 2500)
@@ -38,7 +38,7 @@
<outbound-resourceadapter>
<connection-definition>
- <managedconnectionfactory-class>org.teiid.resource.cci.ldap.LDAPManagedConnectionFactory</managedconnectionfactory-class>
+ <managedconnectionfactory-class>org.teiid.resource.adapter.ldap.LDAPManagedConnectionFactory</managedconnectionfactory-class>
<config-property>
<description>{$display:"Ldap Admin User DN",$description:"User DN for the LDAP admin account.",$required:"true"}</description>
13 years, 8 months
- 1
- 0
Teiid-NightlyBuild - Build # 929 - Failure!
by jboss-qa-internal＠redhat.com
Teiid-NightlyBuild - Build # 929 - Failure:
Check console output at http://hudson.qa.jboss.com/hudson/job/Teiid-NightlyBuild/929/ to view the results.
13 years, 8 months
- 1
- 1
teiid SVN: r2499 - branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content.
by teiid-commits＠lists.jboss.org
Author: rareddy
Date: 2010-08-27 17:20:51 -0400 (Fri, 27 Aug 2010)
New Revision: 2499
Modified:
branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml
branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/odbc.xml
Log:
misc doc edits.
Modified: branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml
===================================================================
--- branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml 2010-08-27 21:19:35 UTC (rev 2498)
+++ branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/jdbc-connection.xml 2010-08-27 21:20:51 UTC (rev 2499)
@@ -407,7 +407,7 @@
directory. Based on the type of deployment (XA, driver, or local), the contents of the file will be different. See the following sections for more.</para></listitem>
</orderedlist>
- <para>Tje data source will then be accessable through the JNDI name specified in the -ds.xml file.</para>
+ <para>The data source will then be accessable through the JNDI name specified in the -ds.xml file.</para>
<section id="as_xa_connection">
<title>DataSource Connection</title>
Modified: branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/odbc.xml
===================================================================
--- branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/odbc.xml 2010-08-27 21:19:35 UTC (rev 2498)
+++ branches/7.1.x/documentation/client-developers-guide/src/main/docbook/en-US/content/odbc.xml 2010-08-27 21:20:51 UTC (rev 2499)
@@ -206,7 +206,7 @@
ReadOnly = no
ServerType = Postgres
ConnSettings =
- UseServerSidePrepare=1
+ UseServerSidePrepare=0
ByteaAsLongVarBinary=1
Optimizer=0
Ksqo=0
13 years, 8 months
- 1
- 0
teiid SVN: r2498 - branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content.
by teiid-commits＠lists.jboss.org
Author: rareddy
Date: 2010-08-27 17:19:35 -0400 (Fri, 27 Aug 2010)
New Revision: 2498
Modified:
branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content/security.xml
Log:
TEIID-1048: adding doc about how to configure Teiid for pass-through auth.
Modified: branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content/security.xml
===================================================================
--- branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content/security.xml 2010-08-27 19:00:01 UTC (rev 2497)
+++ branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content/security.xml 2010-08-27 21:19:35 UTC (rev 2498)
@@ -10,7 +10,8 @@
<para>Typically a user name is required, however user names may be considered optional if the
identity of the user can be discerned by the password credential alone. In
any case it is up to the configured security domain to determine whether a user can be
- authenticated.</para>
+ authenticated. If you need authentication, the administrator must configure a LoginModule to be used with Teiid.
+ See below for more information on how configure the Login module in JBoss AS.</para>
<note><para>By default, access to Teiid is NOT secure. The default login modules are only
backed by file based authentication, which has a well known user
@@ -18,6 +19,18 @@
The same is true for making connections to the Admin Console application.
We DO NOT recommend leaving the default security profile as defined when you
are exposing sensitive data.</para></note>
+ <section>
+ <title>Pass-through Authentication</title>
+ <para>If your client application (web application or Web service) resides in the same JBoss AS instance as Teiid and
+ client application uses a security-domain to handle the security concerns, then you can configure Teiid to use the
+ same security-domain and not force the user to re-authenticate for using Teiid. In this case Teiid looks for a authenticated
+ subject in the calling thread context and uses for its session and authorization purposes. To configure Teiid for this
+ pass-through authentication mechanism, you need change the Teiid's security-domain name to same name as your
+ application's security domain name in the "teiid-jboss-beans.xml" file in the SessionService section.
+ Please note that for this to work, the security-domain
+ must be a JAAS based Login Module and your client application MUST obtain Teiid connection
+ using <emphasis>Local</emphasis> Connection.</para>
+ </section>
</section>
<section>
<title>Authorization</title>
13 years, 8 months
- 1
- 0
teiid SVN: r2497 - in branches/7.1.x: documentation/caching-guide/src/main/docbook/en-US/content and 6 other directories.
by teiid-commits＠lists.jboss.org
Author: shawkins
Date: 2010-08-27 15:00:01 -0400 (Fri, 27 Aug 2010)
New Revision: 2497
Added:
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java
Removed:
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexCondition.java
Modified:
branches/7.1.x/client/src/main/java/org/teiid/client/plan/Annotation.java
branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml
branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/STree.java
branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java
branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java
branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java
branches/7.1.x/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTableDataManager.java
branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java
branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java
Log:
TEIID-1210 adding support for secondary indexes for materialized views
Modified: branches/7.1.x/client/src/main/java/org/teiid/client/plan/Annotation.java
===================================================================
--- branches/7.1.x/client/src/main/java/org/teiid/client/plan/Annotation.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/client/src/main/java/org/teiid/client/plan/Annotation.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -33,6 +33,7 @@
public class Annotation implements Externalizable {
public static final String MATERIALIZED_VIEW = "Materialized View"; //$NON-NLS-1$
+ public static final String CACHED_PROCEDURE = "Cached Procedure"; //$NON-NLS-1$
public static final String HINTS = "Hints"; //$NON-NLS-1$
public enum Priority {
Modified: branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml
===================================================================
--- branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml 2010-08-27 19:00:01 UTC (rev 2497)
@@ -46,14 +46,22 @@
<note><para>It is important to ensure that all key/index information is present as these will be used by the materialization process to enhance the performance of the materialized table.</para></note>
<para>The target materialized table may also be set in the properties. If the value is left blank, the default, then internal materialization will be used.
Otherwise for external materialization, the value should reference the fully qualified name of a table (or possibly view) with the same columns as the materialized view.
- For most basic scenarios the simplicity of internal materialization makes it the more appealing option. Other considerations for chosing between internal and external materialization are:
+ For most basic scenarios the simplicity of internal materialization makes it the more appealing option.
<itemizedlist>
- <listitem><para>Does the cached data need to be fully durable? If yes, then external materialization should be used.
+ <title>Reasons to use external materialization</title>
+ <listitem><para>The cached data needs to be fully durable.
Internal materialization should not survive a cluster restart.</para>
</listitem>
- <listitem><para>Is full control needed of loading and refresh? If yes, then external materialzation should be used.
+ <listitem><para>Full control is needed of loading and refresh.
Internal materialization does offer several system supported methods for refreshing, but does not give full access to the materialized table.</para>
</listitem>
+ <listitem><para>Control is needed over the materialized table definition.
+ Internal materialization does support <link linkend="internal-index">secondary indexes</link>, but they cannot be directly controlled.
+ Constraints or other database features cannot be added to internal materialization tables.</para>
+ </listitem>
+ <listitem><para>The data volume is large. Internal materialization (and temp tables in general) have memory overhead for each page.
+ A rough guideline is that no more than 100 million rows should in all materializated tables across all VDBs for every 1 gigabyte of heap.</para>
+ </listitem>
</itemizedlist>
</para>
<note><para>Materialized view tables are always scoped to the VDB. If a materialized view definition directly or transitively contains a non-deterministic
@@ -83,7 +91,7 @@
<listitem>
<para>Determine a load and refresh strategy. With the schema created the most simplistic approach is to just load the data.
The load can even be done through Teiid with <code>insert into target_table select * from matview option nocache</code>.
- That however may be too simplistic because you index creation may be more performant if deferred until after the table has been created.
+ That however may be too simplistic because your index creation may be more performant if deferred until after the table has been created.
Also full snapshot refreshes are best done to a staging table then swapping it for the existing physical table to ensure that the refresh
does not impact user queries and to ensure that the table is valid prior to use.</para>
</listitem>
@@ -94,7 +102,8 @@
<para>Internal materialization creates Teiid temporary tables to hold the materialized table. While these tables are not fully durable, they perform
well in most circumstances and the data is present at each Teiid instance which removes the single point of failure and network overhead of an external database.
Internal materialization also provides more built-in facilities for refreshing and monitoring.</para>
- <para>The cache hint, when used in the context of an internal materialized view transformation query, provides the ability to fine tune the materializated table. The pref_mem option also applies to internal materialized views. Internal table index pages already have a memory preference, so the perf_mem option indicates that the data pages should prefer memory as well.</para>
+ <para>The cache hint, when used in the context of an internal materialized view transformation query, provides the ability to fine tune the materializated table.
+ The pref_mem option also applies to internal materialized views. Internal table index pages already have a memory preference, so the perf_mem option indicates that the data pages should prefer memory as well.</para>
<section>
<title>Loading And Refreshing</title>
<para>An internal materialized view table is initially in an invalid state (there is no data). The first user query will trigger an implicit loading of the data.
@@ -154,10 +163,8 @@
</section>
</section>
<section>
- <title>Limitations</title>
- <itemizedlist>
- <listitem><para>Secondary index information is currently not used. An index is only created for the primary key.</para></listitem>
- </itemizedlist>
+ <title>Secondary Indexes</title>
+ <para>Function based indexes, Covering indexes, converting unique to index</para>
</section>
</section>
</chapter>
Modified: branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/STree.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -392,4 +392,8 @@
this.preferMemory = preferMemory;
}
+ public boolean isPreferMemory() {
+ return preferMemory;
+ }
+
}
\ No newline at end of file
Modified: branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -33,12 +33,14 @@
/**
* Implements intelligent browsing over a {@link STree}
+ *
+ * TODO: this is not as efficient as it should be over partial matches
*/
public class TupleBrowser implements TupleSource {
private final STree tree;
- private List<List<Object>> valueSet;
+ private TupleSource valueSet;
private SPage page;
private int index;
@@ -53,12 +55,13 @@
private boolean inPartial;
/**
- * Construct a value based browser
+ * Construct a value based browser. The {@link TupleSource} should already be in the
+ * proper direction.
* @param sTree
* @param valueSet
* @param direction
*/
- public TupleBrowser(STree sTree, List<List<Object>> valueSet, boolean direction) {
+ public TupleBrowser(STree sTree, TupleSource valueSet, boolean direction) {
this.tree = sTree;
this.direction = direction;
this.valueSet = valueSet;
@@ -80,7 +83,7 @@
}
private void init(List<Object> lowerBound,
- List<Object> upperBound)
+ List<?> upperBound)
throws TeiidComponentException {
if (lowerBound != null) {
lowerBound.addAll(Collections.nCopies(tree.getKeyLength() - lowerBound.size(), null));
@@ -156,11 +159,11 @@
for (;;) {
//first check for value iteration
if (!inPartial && valueSet != null) {
- if (valueSet.isEmpty()) {
+ List<?> newValue = valueSet.nextTuple();
+ if (newValue == null) {
resetState();
return null;
}
- List<Object> newValue = direction?valueSet.remove(0):valueSet.remove(valueSet.size() -1);
if (newValue.size() < tree.getKeyLength()) {
init(new ArrayList<Object>(newValue), newValue);
inPartial = true;
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataAdapter.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -23,9 +23,9 @@
package org.teiid.query.metadata;
import java.util.ArrayList;
-import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Properties;
@@ -363,7 +363,11 @@
public Collection getIndexesInGroup(Object groupID)
throws TeiidComponentException, QueryMetadataException {
if(groupID instanceof TempMetadataID) {
- return Collections.EMPTY_LIST;
+ List<List<TempMetadataID>> result = ((TempMetadataID)groupID).getIndexes();
+ if (result == null) {
+ return Collections.emptyList();
+ }
+ return result;
}
return this.actualMetadata.getIndexesInGroup(groupID);
}
@@ -372,11 +376,15 @@
throws TeiidComponentException, QueryMetadataException {
if(groupID instanceof TempMetadataID) {
+ LinkedList<List<TempMetadataID>> result = new LinkedList<List<TempMetadataID>>();
TempMetadataID id = (TempMetadataID)groupID;
- if (id.getPrimaryKey() == null) {
- return Collections.emptyList();
+ if (id.getPrimaryKey() != null) {
+ result.add(id.getPrimaryKey());
}
- return Arrays.asList(groupID);
+ if (id.getUniqueKeys() != null) {
+ result.addAll(id.getUniqueKeys());
+ }
+ return result;
}
return this.actualMetadata.getUniqueKeysInGroup(groupID);
}
@@ -401,8 +409,8 @@
public List getElementIDsInKey(Object keyID)
throws TeiidComponentException, QueryMetadataException {
- if (keyID instanceof TempMetadataID) {
- return ((TempMetadataID)keyID).getPrimaryKey();
+ if (keyID instanceof List) {
+ return (List)keyID;
}
return this.actualMetadata.getElementIDsInKey(keyID);
@@ -692,10 +700,7 @@
@Override
public Object getPrimaryKey(Object metadataID) {
if (metadataID instanceof TempMetadataID) {
- if (((TempMetadataID)metadataID).getPrimaryKey() != null) {
- return metadataID;
- }
- return null;
+ return ((TempMetadataID)metadataID).getPrimaryKey();
}
return this.actualMetadata.getPrimaryKey(metadataID);
}
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/metadata/TempMetadataID.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -25,6 +25,7 @@
import java.io.Serializable;
import java.util.Collection;
import java.util.Collections;
+import java.util.LinkedList;
import java.util.List;
import org.teiid.core.util.LRUCache;
@@ -37,11 +38,28 @@
* does not exist in a real metadata source. Rather, it is used temporarily
* in context of processing a single query. This metadata ID can be used to
* represent either a group or an element depending on the constructor used.
+ *
+ * TODO: we should be using the real metadata objects, but internal and
+ * designer legacy keep us on the temp framework
*/
public class TempMetadataID implements Serializable {
private static final int LOCAL_CACHE_SIZE = 8;
+ static class TableData {
+ Collection<TempMetadataID> accessPatterns;
+ List<TempMetadataID> elements;
+ int cardinality = QueryMetadataInterface.UNKNOWN_CARDINALITY;
+ List<TempMetadataID> primaryKey;
+ QueryNode queryNode;
+ LRUCache<Object, Object> localCache;
+ CacheHint cacheHint;
+ List<List<TempMetadataID>> keys;
+ List<List<TempMetadataID>> indexes;
+ }
+
+ private static TableData DUMMY_DATA = new TableData();
+
public enum Type {
VIRTUAL,
TEMP,
@@ -50,18 +68,11 @@
private String ID; // never null, upper cased fully-qualified string
private Type metadataType = Type.VIRTUAL;
+ private Object originalMetadataID;
- //Table metadata
- private Collection<TempMetadataID> accessPatterns;
- private List<TempMetadataID> elements; // of TempMetadataID, only for group
- private int cardinality = QueryMetadataInterface.UNKNOWN_CARDINALITY;
- private List<TempMetadataID> primaryKey;
- private QueryNode queryNode;
- private transient LRUCache<Object, Object> localCache;
- private CacheHint cacheHint;
+ private TableData data;
- //Column metadata
- private Object originalMetadataID;
+ //Column metadata
private int position;
private Class<?> type; // type of this element, only for element
@@ -81,8 +92,9 @@
* @param isVirtual whether or not the group is a virtual group
*/
public TempMetadataID(String ID, List<TempMetadataID> elements, Type type) {
+ this.data = new TableData();
this.ID = ID;
- this.elements = elements;
+ this.data.elements = elements;
int pos = 1;
for (TempMetadataID tempMetadataID : elements) {
tempMetadataID.setPosition(pos++);
@@ -133,7 +145,7 @@
* @return List of TempMetadataID for groups, null for elements
*/
public List<TempMetadataID> getElements() {
- return this.elements;
+ return this.getTableData().elements;
}
/**
@@ -141,12 +153,12 @@
* @param elem
*/
protected void addElement(TempMetadataID elem) {
- if (this.elements != null) {
- this.elements.add(elem);
- elem.setPosition(this.elements.size());
+ if (this.getTableData().elements != null) {
+ this.getTableData().elements.add(elem);
+ elem.setPosition(this.getTableData().elements.size());
}
- if (this.localCache != null) {
- this.localCache.clear();
+ if (this.getTableData().localCache != null) {
+ this.getTableData().localCache.clear();
}
}
@@ -198,7 +210,7 @@
return this.ID.hashCode();
}
- public void setOriginalMetadataID(Object metadataId) {
+ public void setOriginalMetadataID(Object metadataId) {
this.originalMetadataID = metadataId;
}
@@ -211,22 +223,22 @@
}
public Collection<TempMetadataID> getAccessPatterns() {
- if (this.accessPatterns == null) {
+ if (this.getTableData().accessPatterns == null) {
return Collections.emptyList();
}
- return this.accessPatterns;
+ return this.getTableData().accessPatterns;
}
public void setAccessPatterns(Collection<TempMetadataID> accessPatterns) {
- this.accessPatterns = accessPatterns;
+ this.getTableData().accessPatterns = accessPatterns;
}
public int getCardinality() {
- return this.cardinality;
+ return this.getTableData().cardinality;
}
public void setCardinality(int cardinality) {
- this.cardinality = cardinality;
+ this.getTableData().cardinality = cardinality;
}
public void setTempTable(boolean isTempTable) {
@@ -238,17 +250,17 @@
}
Object getProperty(Object key) {
- if (this.localCache != null) {
- return this.localCache.get(key);
+ if (this.getTableData().localCache != null) {
+ return this.getTableData().localCache.get(key);
}
return null;
}
Object setProperty(Object key, Object value) {
- if (this.localCache == null) {
- this.localCache = new LRUCache<Object, Object>(LOCAL_CACHE_SIZE);
+ if (this.getTableData().localCache == null) {
+ this.getTableData().localCache = new LRUCache<Object, Object>(LOCAL_CACHE_SIZE);
}
- return this.localCache.put(key, value);
+ return this.getTableData().localCache.put(key, value);
}
public boolean isScalarGroup() {
@@ -260,11 +272,11 @@
}
public List<TempMetadataID> getPrimaryKey() {
- return primaryKey;
+ return getTableData().primaryKey;
}
public void setPrimaryKey(List<TempMetadataID> primaryKey) {
- this.primaryKey = primaryKey;
+ this.getTableData().primaryKey = primaryKey;
}
public int getPosition() {
@@ -276,19 +288,48 @@
}
public QueryNode getQueryNode() {
- return queryNode;
+ return getTableData().queryNode;
}
public void setQueryNode(QueryNode queryNode) {
- this.queryNode = queryNode;
+ this.getTableData().queryNode = queryNode;
}
public CacheHint getCacheHint() {
- return cacheHint;
+ return getTableData().cacheHint;
}
public void setCacheHint(CacheHint cacheHint) {
- this.cacheHint = cacheHint;
+ this.getTableData().cacheHint = cacheHint;
}
+
+ public List<List<TempMetadataID>> getIndexes() {
+ return getTableData().indexes;
+ }
+
+ public void addIndex(List<TempMetadataID> index) {
+ if (this.getTableData().indexes == null) {
+ this.getTableData().indexes = new LinkedList<List<TempMetadataID>>();
+ }
+ this.getTableData().indexes.add(index);
+ }
+
+ public List<List<TempMetadataID>> getUniqueKeys() {
+ return getTableData().keys;
+ }
+
+ public void addUniqueKey(List<TempMetadataID> key) {
+ if (this.getTableData().keys == null) {
+ this.getTableData().keys = new LinkedList<List<TempMetadataID>>();
+ }
+ this.getTableData().keys.add(key);
+ }
+
+ private TableData getTableData() {
+ if (data == null) {
+ return DUMMY_DATA;
+ }
+ return data;
+ }
}
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/optimizer/relational/RelationalPlanner.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -490,12 +490,12 @@
if (container.areResultsCachable() && Query.areResultsCachable(container.getProcedureParameters().keySet()) && context.isResultSetCacheEnabled()) {
container.getGroup().setGlobalTable(true);
container.setCacheHint(c.getCacheHint());
- recordAnnotation(analysisRecord, Annotation.MATERIALIZED_VIEW, Priority.LOW, "SimpleQueryResolver.procedure_cache_used", container.getGroup()); //$NON-NLS-1$
+ recordAnnotation(analysisRecord, Annotation.CACHED_PROCEDURE, Priority.LOW, "SimpleQueryResolver.procedure_cache_used", container.getGroup()); //$NON-NLS-1$
return;
}
- recordAnnotation(analysisRecord, Annotation.MATERIALIZED_VIEW, Priority.MEDIUM, "SimpleQueryResolver.procedure_cache_not_usable", container.getGroup()); //$NON-NLS-1$
+ recordAnnotation(analysisRecord, Annotation.CACHED_PROCEDURE, Priority.MEDIUM, "SimpleQueryResolver.procedure_cache_not_usable", container.getGroup()); //$NON-NLS-1$
} else {
- recordAnnotation(analysisRecord, Annotation.MATERIALIZED_VIEW, Priority.LOW, "SimpleQueryResolver.procedure_cache_not_used", container.getGroup()); //$NON-NLS-1$
+ recordAnnotation(analysisRecord, Annotation.CACHED_PROCEDURE, Priority.LOW, "SimpleQueryResolver.procedure_cache_not_used", container.getGroup()); //$NON-NLS-1$
}
}
}
@@ -984,16 +984,18 @@
id.setCardinality(metadata.getCardinality(table.getMetadataID()));
Object pk = metadata.getPrimaryKey(table.getMetadataID());
- //primary key
if (pk != null) {
- List cols = metadata.getElementIDsInKey(pk);
- ArrayList<TempMetadataID> primaryKey = new ArrayList<TempMetadataID>(cols.size());
- for (Object coldId : cols) {
- int pos = metadata.getPosition(coldId) - 1;
- primaryKey.add(id.getElements().get(pos));
- }
+ ArrayList<TempMetadataID> primaryKey = resolveIndex(metadata, id, pk);
id.setPrimaryKey(primaryKey);
}
+ Collection keys = metadata.getUniqueKeysInGroup(table.getMetadataID());
+ for (Object key : keys) {
+ id.addUniqueKey(resolveIndex(metadata, id, key));
+ }
+ Collection indexes = metadata.getIndexesInGroup(table.getMetadataID());
+ for (Object index : indexes) {
+ id.addIndex(resolveIndex(metadata, id, index));
+ }
Command c = getCommand(table, metadata.getVirtualPlan(table.getMetadataID()), SQLConstants.Reserved.SELECT, metadata, analysisRecord);
CacheHint hint = c.getCacheHint();
if (hint != null) {
@@ -1008,6 +1010,18 @@
return id;
}
+ private static ArrayList<TempMetadataID> resolveIndex(
+ QueryMetadataInterface metadata, TempMetadataID id, Object pk)
+ throws TeiidComponentException, QueryMetadataException {
+ List cols = metadata.getElementIDsInKey(pk);
+ ArrayList<TempMetadataID> primaryKey = new ArrayList<TempMetadataID>(cols.size());
+ for (Object coldId : cols) {
+ int pos = metadata.getPosition(coldId) - 1;
+ primaryKey.add(id.getElements().get(pos));
+ }
+ return primaryKey;
+ }
+
private static Command getCommand(GroupSymbol virtualGroup, QueryNode qnode,
String cacheString, QueryMetadataInterface qmi, AnalysisRecord analysisRecord) throws TeiidComponentException,
QueryMetadataException, QueryResolverException,
Deleted: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexCondition.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexCondition.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexCondition.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -1,152 +0,0 @@
-/*
- * JBoss, Home of Professional Open Source.
- * See the COPYRIGHT.txt file distributed with this work for information
- * regarding copyright ownership. Some portions may be licensed
- * to Red Hat, Inc. under one or more contributor license agreements.
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * Lesser General Public License for more details.
- *
- * You should have received a copy of the GNU Lesser General Public
- * License along with this library; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- * 02110-1301 USA.
- */
-
-package org.teiid.query.tempdata;
-
-import java.util.Iterator;
-import java.util.List;
-import java.util.TreeSet;
-
-import org.teiid.query.sql.lang.CompareCriteria;
-import org.teiid.query.sql.lang.Criteria;
-import org.teiid.query.sql.lang.IsNullCriteria;
-import org.teiid.query.sql.lang.MatchCriteria;
-import org.teiid.query.sql.lang.SetCriteria;
-import org.teiid.query.sql.symbol.Constant;
-import org.teiid.query.sql.symbol.ElementSymbol;
-
-/**
- * Accumulates information from criteria about a specific index column.
- */
-class IndexCondition {
-
- static IndexCondition[] getIndexConditions(Criteria condition, List<ElementSymbol> keyColumns) {
- List<Criteria> crits = Criteria.separateCriteriaByAnd(condition);
- IndexCondition[] conditions = new IndexCondition[keyColumns.size()];
- for (int i = 0; i < conditions.length; i++) {
- conditions[i] = new IndexCondition();
- ElementSymbol keyColumn = keyColumns.get(i);
- for (Iterator<Criteria> critIter = crits.iterator(); critIter.hasNext();) {
- Criteria criteria = critIter.next();
- if (criteria instanceof CompareCriteria) {
- CompareCriteria cc = (CompareCriteria)criteria;
- if (cc.getOperator() == CompareCriteria.NE
- || !(cc.getRightExpression() instanceof Constant)) {
- critIter.remove();
- continue;
- }
- if (!cc.getLeftExpression().equals(keyColumn)) {
- continue;
- }
- conditions[i].addCondition((Constant)cc.getRightExpression(), cc.getOperator());
- critIter.remove();
- } else if (criteria instanceof IsNullCriteria) {
- IsNullCriteria inc = (IsNullCriteria)criteria;
- if (inc.isNegated() || !inc.getExpression().equals(keyColumn)) {
- continue;
- }
- conditions[i].addCondition(new Constant(null), CompareCriteria.EQ);
- critIter.remove();
- } else {
- if (i > 0) {
- critIter.remove();
- continue;
- }
- if (criteria instanceof MatchCriteria) {
- MatchCriteria matchCriteria = (MatchCriteria)criteria;
- if (matchCriteria.isNegated() || !matchCriteria.getLeftExpression().equals(keyColumn) || !(matchCriteria.getRightExpression() instanceof Constant)) {
- continue;
- }
- Constant value = (Constant)matchCriteria.getRightExpression();
- String pattern = (String)value.getValue();
- boolean escaped = false;
- StringBuilder prefix = new StringBuilder();
- for (int j = 0; i < pattern.length(); j++) {
- char character = pattern.charAt(j);
-
- if (character == matchCriteria.getEscapeChar() && character != MatchCriteria.NULL_ESCAPE_CHAR) {
- if (escaped) {
- prefix.append(character);
- escaped = false;
- } else {
- escaped = true;
- }
- } else if (character == MatchCriteria.WILDCARD_CHAR || character == MatchCriteria.MATCH_CHAR) {
- break;
- } else {
- prefix.append(character);
- }
- }
- if (prefix.length() > 0) {
- conditions[i].addCondition(new Constant(prefix.toString()), CompareCriteria.GE);
- }
- } else if (criteria instanceof SetCriteria) {
- SetCriteria setCriteria = (SetCriteria)criteria;
- if (!setCriteria.getExpression().equals(keyColumn) || !setCriteria.isAllConstants()) {
- continue;
- }
- TreeSet<Constant> values = (TreeSet<Constant>) setCriteria.getValues();
- conditions[i].addSet(values);
- }
- }
- }
- }
- return conditions;
- }
-
- Constant lower = null;
- Constant upper = null;
- TreeSet<Constant> valueSet = new TreeSet<Constant>();
-
- void addCondition(Constant value, int comparisionMode) {
- switch (comparisionMode) {
- case CompareCriteria.EQ:
- valueSet.clear();
- valueSet.add(value);
- lower = null;
- upper = null;
- break;
- case CompareCriteria.GE:
- case CompareCriteria.GT:
- if (valueSet.isEmpty()) {
- lower = value;
- }
- break;
- case CompareCriteria.LE:
- case CompareCriteria.LT:
- if (valueSet.isEmpty()) {
- upper = value;
- }
- break;
- }
- }
-
- void addSet(TreeSet<Constant> values) {
- if (!valueSet.isEmpty()) {
- return;
- }
- lower = null;
- upper = null;
- valueSet.addAll(values);
- }
-
-}
\ No newline at end of file
Copied: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java (from rev 2478, branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexCondition.java)
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java (rev 0)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -0,0 +1,263 @@
+/*
+ * JBoss, Home of Professional Open Source.
+ * See the COPYRIGHT.txt file distributed with this work for information
+ * regarding copyright ownership. Some portions may be licensed
+ * to Red Hat, Inc. under one or more contributor license agreements.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301 USA.
+ */
+
+package org.teiid.query.tempdata;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import java.util.ListIterator;
+import java.util.TreeSet;
+
+import org.teiid.common.buffer.TupleBrowser;
+import org.teiid.common.buffer.TupleSource;
+import org.teiid.core.TeiidComponentException;
+import org.teiid.query.processor.CollectionTupleSource;
+import org.teiid.query.processor.relational.RelationalNode;
+import org.teiid.query.sql.lang.CompareCriteria;
+import org.teiid.query.sql.lang.Criteria;
+import org.teiid.query.sql.lang.IsNullCriteria;
+import org.teiid.query.sql.lang.MatchCriteria;
+import org.teiid.query.sql.lang.OrderBy;
+import org.teiid.query.sql.lang.OrderByItem;
+import org.teiid.query.sql.lang.SetCriteria;
+import org.teiid.query.sql.symbol.Constant;
+import org.teiid.query.sql.symbol.ElementSymbol;
+import org.teiid.query.sql.symbol.SingleElementSymbol;
+
+/**
+ * Accumulates information about index usage.
+ */
+class IndexInfo {
+
+ List<Object> lower = null;
+ List<Object> upper = null;
+ ArrayList<List<Object>> valueSet = new ArrayList<List<Object>>();
+ TempTable table;
+ Boolean ordering;
+ boolean covering;
+ TupleSource valueTs;
+
+ public IndexInfo(TempTable table, final List<? extends SingleElementSymbol> projectedCols, final Criteria condition, OrderBy orderBy, boolean primary) {
+ this.table = table;
+ if (primary || this.table.getColumnMap().keySet().containsAll(projectedCols)) {
+ covering = true;
+ }
+ if (table.getPkLength() > 0) {
+ processCriteria(condition);
+ if (orderBy != null && (covering || this.table.getColumnMap().keySet().containsAll(orderBy.getSortKeys()))) {
+ ordering = useIndexForOrderBy(orderBy);
+ }
+ }
+ }
+
+ private void processCriteria(Criteria condition) {
+ List<Criteria> crits = Criteria.separateCriteriaByAnd(condition);
+ for (int i = 0; i < table.getPkLength(); i++) {
+ ElementSymbol keyColumn = table.getColumns().get(i);
+ for (Iterator<Criteria> critIter = crits.iterator(); critIter.hasNext();) {
+ Criteria criteria = critIter.next();
+ if (criteria instanceof CompareCriteria) {
+ CompareCriteria cc = (CompareCriteria)criteria;
+ if (cc.getOperator() == CompareCriteria.NE
+ || !(cc.getRightExpression() instanceof Constant)) {
+ critIter.remove();
+ continue;
+ }
+ if (!cc.getLeftExpression().equals(keyColumn)) {
+ continue;
+ }
+ this.addCondition(i, (Constant)cc.getRightExpression(), cc.getOperator());
+ critIter.remove();
+ } else if (criteria instanceof IsNullCriteria) {
+ IsNullCriteria inc = (IsNullCriteria)criteria;
+ if (inc.isNegated() || !inc.getExpression().equals(keyColumn)) {
+ continue;
+ }
+ this.addCondition(i, new Constant(null), CompareCriteria.EQ);
+ critIter.remove();
+ } else {
+ if (i > 0) {
+ critIter.remove();
+ continue;
+ }
+ if (criteria instanceof MatchCriteria) {
+ MatchCriteria matchCriteria = (MatchCriteria)criteria;
+ if (matchCriteria.isNegated() || !matchCriteria.getLeftExpression().equals(keyColumn) || !(matchCriteria.getRightExpression() instanceof Constant)) {
+ continue;
+ }
+ Constant value = (Constant)matchCriteria.getRightExpression();
+ String pattern = (String)value.getValue();
+ boolean escaped = false;
+ StringBuilder prefix = new StringBuilder();
+ for (int j = 0; i < pattern.length(); j++) {
+ char character = pattern.charAt(j);
+
+ if (character == matchCriteria.getEscapeChar() && character != MatchCriteria.NULL_ESCAPE_CHAR) {
+ if (escaped) {
+ prefix.append(character);
+ escaped = false;
+ } else {
+ escaped = true;
+ }
+ } else if (character == MatchCriteria.WILDCARD_CHAR || character == MatchCriteria.MATCH_CHAR) {
+ break;
+ } else {
+ prefix.append(character);
+ }
+ }
+ if (prefix.length() > 0) {
+ this.addCondition(i, new Constant(prefix.toString()), CompareCriteria.GE);
+ }
+ } else if (criteria instanceof SetCriteria) {
+ SetCriteria setCriteria = (SetCriteria)criteria;
+ if (!setCriteria.getExpression().equals(keyColumn) || !setCriteria.isAllConstants()) {
+ continue;
+ }
+ TreeSet<Constant> values = (TreeSet<Constant>) setCriteria.getValues();
+ this.addSet(i, values);
+ }
+ }
+ }
+ }
+ }
+
+ void addCondition(int i, Constant value, int comparisionMode) {
+ switch (comparisionMode) {
+ case CompareCriteria.EQ:
+ if (i == 0) {
+ valueSet.clear();
+ valueSet.add(new ArrayList<Object>(table.getPkLength()));
+ }
+ if (valueSet.size() == 1) {
+ valueSet.get(0).add(value.getValue());
+ }
+ lower = null;
+ upper = null;
+ break;
+ case CompareCriteria.GE:
+ case CompareCriteria.GT:
+ if (valueSet.isEmpty()) {
+ if (i == 0) {
+ lower = new ArrayList<Object>(table.getPkLength());
+ lower.add(value.getValue());
+ } if (lower != null && lower.size() == i) {
+ lower.add(value.getValue());
+ }
+ }
+ break;
+ case CompareCriteria.LE:
+ case CompareCriteria.LT:
+ if (valueSet.isEmpty()) {
+ if (i == 0) {
+ upper = new ArrayList<Object>(table.getPkLength());
+ upper.add(value.getValue());
+ } else if (upper != null && upper.size() == i) {
+ upper.add(value.getValue());
+ }
+ }
+ break;
+ }
+ }
+
+ void addSet(int i, TreeSet<Constant> values) {
+ if (!valueSet.isEmpty()) {
+ return;
+ }
+ if (i == 0) {
+ for (Constant constant : values) {
+ List<Object> value = new ArrayList<Object>(table.getPkLength());
+ value.add(constant.getValue());
+ valueSet.add(value);
+ }
+ lower = null;
+ upper = null;
+ }
+ }
+
+ /**
+ * Return a non-null direction if the index can be used, otherwise null.
+ * @param orderBy
+ * @return
+ */
+ private Boolean useIndexForOrderBy(OrderBy orderBy) {
+ Boolean direction = null;
+ int[] orderByIndexes = RelationalNode.getProjectionIndexes(this.table.getColumnMap(), orderBy.getSortKeys());
+ for (int i = 0; i < table.getPkLength(); i++) {
+ if (orderByIndexes.length <= i) {
+ break;
+ }
+ if (orderByIndexes[i] != i) {
+ return null;
+ }
+ }
+ for (OrderByItem item : orderBy.getOrderByItems()) {
+ if (item.getNullOrdering() != null) {
+ return null;
+ }
+ if (item.isAscending()) {
+ if (direction == null) {
+ direction = OrderBy.ASC;
+ } else if (direction != OrderBy.ASC) {
+ return null;
+ }
+ } else if (direction == null) {
+ direction = OrderBy.DESC;
+ } else if (direction != OrderBy.DESC) {
+ return null;
+ }
+ }
+ return direction;
+ }
+
+ TupleBrowser createTupleBrowser() throws TeiidComponentException {
+ boolean direction = ordering == null ? OrderBy.ASC : ordering;
+ if (valueTs != null) {
+ return new TupleBrowser(this.table.getTree(), valueTs, direction);
+ }
+ if (!valueSet.isEmpty()) {
+ CollectionTupleSource cts = null;
+ if (direction == OrderBy.ASC) {
+ cts = new CollectionTupleSource(valueSet.iterator());
+ } else {
+ cts = new CollectionTupleSource(new Iterator<List<Object>>() {
+ ListIterator<List<Object>> iter = valueSet.listIterator(valueSet.size());
+ @Override
+ public boolean hasNext() {
+ return iter.hasPrevious();
+ }
+ @Override
+ public List<Object> next() {
+ return iter.previous();
+ }
+ @Override
+ public void remove() {
+ throw new UnsupportedOperationException();
+ }
+ });
+ }
+ return new TupleBrowser(this.table.getTree(), cts, direction);
+ }
+ return new TupleBrowser(this.table.getTree(), lower, upper, direction);
+ }
+
+}
\ No newline at end of file
Property changes on: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/IndexInfo.java
___________________________________________________________________
Name: svn:mime-type
+ text/plain
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -23,7 +23,9 @@
package org.teiid.query.tempdata;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashSet;
+import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -56,9 +58,7 @@
import org.teiid.query.sql.lang.CacheHint;
import org.teiid.query.sql.lang.Criteria;
import org.teiid.query.sql.lang.OrderBy;
-import org.teiid.query.sql.lang.OrderByItem;
import org.teiid.query.sql.lang.SetClauseList;
-import org.teiid.query.sql.symbol.Constant;
import org.teiid.query.sql.symbol.ElementSymbol;
import org.teiid.query.sql.symbol.Expression;
import org.teiid.query.sql.symbol.SingleElementSymbol;
@@ -70,6 +70,8 @@
*/
class TempTable {
+ private static final double LN_2 = Math.log(2);
+
private final class InsertUpdateProcessor extends UpdateProcessor {
private boolean addRowId;
@@ -258,13 +260,16 @@
private TempMetadataID tid;
private ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
private boolean updatable = true;
+ private LinkedHashMap<List<ElementSymbol>, TempTable> indexTables;
private int keyBatchSize;
private int leafBatchSize;
+ private Map columnMap;
TempTable(TempMetadataID tid, BufferManager bm, List<ElementSymbol> columns, int primaryKeyLength, String sessionID) {
this.tid = tid;
this.bm = bm;
+ this.columnMap = RelationalNode.createLookupMap(columns);
if (primaryKeyLength == 0) {
ElementSymbol id = new ElementSymbol("rowId"); //$NON-NLS-1$
id.setType(DataTypeManager.DefaultDataClasses.INTEGER);
@@ -280,64 +285,82 @@
this.leafBatchSize = bm.getSchemaSize(columns.subList(0, primaryKeyLength));
}
+ void addIndex(List<ElementSymbol> indexColumns) throws TeiidComponentException, TeiidProcessingException {
+ List<ElementSymbol> keyColumns = columns.subList(0, tree.getKeyLength());
+ if (keyColumns.equals(indexColumns) || (indexTables != null && indexTables.containsKey(indexColumns))) {
+ return;
+ }
+ List<ElementSymbol> allColumns = new ArrayList<ElementSymbol>(indexColumns);
+ for (ElementSymbol elementSymbol : keyColumns) {
+ if (allColumns.indexOf(elementSymbol) < 0) {
+ allColumns.add(elementSymbol);
+ }
+ }
+ TempTable indexTable = new TempTable(new TempMetadataID("idx", Collections.EMPTY_LIST), this.bm, allColumns, indexColumns.size(), this.sessionID); //$NON-NLS-1$
+ indexTable.setPreferMemory(this.tree.isPreferMemory());
+ if (indexTables == null) {
+ indexTables = new LinkedHashMap<List<ElementSymbol>, TempTable>();
+ indexTables.put(indexColumns, indexTable);
+ }
+ //TODO: ordered insert optimization
+ TupleSource ts = createTupleSource(allColumns, null, null);
+ indexTable.insert(ts, allColumns);
+ }
private int reserveBuffers() {
return bm.reserveBuffers(leafBatchSize + (tree.getHeight() - 1)*keyBatchSize, BufferReserveMode.WAIT);
}
public TupleSource createTupleSource(final List<? extends SingleElementSymbol> projectedCols, final Criteria condition, OrderBy orderBy) throws TeiidComponentException, TeiidProcessingException {
- Map map = RelationalNode.createLookupMap(getColumns());
-
- Boolean direction = null;
- boolean orderByUsingIndex = false;
- if (orderBy != null && rowId == null) {
- int[] orderByIndexes = RelationalNode.getProjectionIndexes(map, orderBy.getSortKeys());
- orderByUsingIndex = true;
- for (int i = 0; i < tree.getKeyLength(); i++) {
- if (orderByIndexes.length <= i) {
- break;
+ IndexInfo primary = new IndexInfo(this, projectedCols, condition, orderBy, true);
+ IndexInfo ii = primary;
+ if (indexTables != null && (condition != null || orderBy != null) && ii.valueSet.size() != 1) {
+ int rowCost = this.tree.getRowCount();
+ int bestCost = estimateCost(orderBy, ii, rowCost);
+ for (TempTable table : this.indexTables.values()) {
+ IndexInfo secondary = new IndexInfo(table, projectedCols, condition, orderBy, false);
+ int cost = estimateCost(orderBy, secondary, rowCost);
+ if (cost < bestCost) {
+ ii = secondary;
+ bestCost = cost;
}
- if (orderByIndexes[i] != i) {
- orderByUsingIndex = false;
- break;
- }
}
- if (orderByUsingIndex) {
- for (OrderByItem item : orderBy.getOrderByItems()) {
- if (item.getNullOrdering() != null) {
- orderByUsingIndex = false;
- break;
- }
- if (item.isAscending()) {
- if (direction == null) {
- direction = OrderBy.ASC;
- } else if (direction != OrderBy.ASC) {
- orderByUsingIndex = false;
- break;
- }
- } else if (direction == null) {
- direction = OrderBy.DESC;
- } else if (direction != OrderBy.DESC) {
- orderByUsingIndex = false;
- break;
- }
- }
+ if (ii.covering) {
+ return ii.table.createTupleSource(projectedCols, condition, orderBy, ii);
}
+ List<ElementSymbol> pkColumns = this.columns.subList(0, this.getPkLength());
+ if (ii.ordering != null) {
+ //use order and join
+ primary.valueTs = ii.table.createTupleSource(pkColumns, condition, orderBy, ii);
+ primary.ordering = null;
+ return createTupleSource(projectedCols, condition, null, primary);
+ }
+ //order by pk to localize lookup costs, then join
+ OrderBy pkOrderBy = new OrderBy();
+ for (ElementSymbol elementSymbol : pkColumns) {
+ pkOrderBy.addVariable(elementSymbol);
+ }
+ primary.valueTs = ii.table.createTupleSource(pkColumns, condition, pkOrderBy, ii);
+ return createTupleSource(projectedCols, condition, orderBy, primary);
}
- if (!orderByUsingIndex) {
- direction = OrderBy.ASC;
- }
- TupleBrowser browser = createTupleBrower(condition, direction);
- TupleSource ts = new QueryTupleSource(browser, map, projectedCols, condition);
+ return createTupleSource(projectedCols, condition, orderBy, ii);
+ }
+
+ private TupleSource createTupleSource(
+ final List<? extends SingleElementSymbol> projectedCols,
+ final Criteria condition, OrderBy orderBy, IndexInfo ii)
+ throws TeiidComponentException, TeiidProcessingException {
+ TupleBrowser browser = ii.createTupleBrowser();
+ TupleSource ts = new QueryTupleSource(browser, columnMap, projectedCols, condition);
boolean usingQueryTupleSource = false;
try {
TupleBuffer tb = null;
- if (!orderByUsingIndex && orderBy != null) {
+ if (ii.ordering == null && orderBy != null) {
SortUtility sort = new SortUtility(ts, orderBy.getOrderByItems(), Mode.SORT, bm, sessionID, projectedCols);
tb = sort.sort();
- } else if (!updatable) {
+ } else if (updatable) {
tb = bm.createTupleBuffer(projectedCols, sessionID, TupleSourceType.PROCESSOR);
List<?> next = null;
while ((next = ts.nextTuple()) != null) {
@@ -357,49 +380,32 @@
}
}
- private TupleBrowser createTupleBrower(Criteria condition, boolean direction) throws TeiidComponentException {
- List<Object> lower = null;
- List<Object> upper = null;
- List<List<Object>> values = null;
- if (condition != null && rowId == null) {
- IndexCondition[] indexConditions = IndexCondition.getIndexConditions(condition, columns.subList(0, tree.getKeyLength()));
- for (int i = 0; i < indexConditions.length; i++) {
- IndexCondition indexCondition = indexConditions[i];
- if (indexCondition.lower != null) {
- if (i == 0) {
- lower = new ArrayList<Object>(tree.getKeyLength());
- lower.add(indexCondition.lower.getValue());
- } if (lower != null && lower.size() == i) {
- lower.add(indexCondition.lower.getValue());
- }
- }
- if (indexCondition.upper != null) {
- if (i == 0) {
- upper = new ArrayList<Object>(tree.getKeyLength());
- upper.add(indexCondition.upper.getValue());
- } else if (upper != null && upper.size() == i) {
- upper.add(indexCondition.upper.getValue());
- }
- }
- if (!indexCondition.valueSet.isEmpty()) {
- if (i == 0) {
- values = new ArrayList<List<Object>>();
- for (Constant constant : indexCondition.valueSet) {
- List<Object> value = new ArrayList<Object>(tree.getKeyLength());
- value.add(constant.getValue());
- values.add(value);
- }
- } else if (values != null && values.size() == 1 && values.iterator().next().size() == i && indexCondition.valueSet.size() == 1) {
- values.iterator().next().add(indexCondition.valueSet.first().getValue());
- }
- }
- }
+ /**
+ * TODO: this could easily use statistics - the tree level 1 would be an ideal place
+ * to compute them, since it minimizes page loads, and is a random sample.
+ * @return
+ */
+ private int estimateCost(OrderBy orderBy, IndexInfo ii, int rowCost) {
+ if (ii.valueSet.size() != 0) {
+ int length = ii.valueSet.get(0).size();
+ rowCost = Math.min(rowCost, ii.valueSet.size() * 1 << (ii.table.getPkLength() - length));
+ } else if (ii.upper != null) {
+ rowCost /= 3;
+ } else if (ii.lower != null) {
+ rowCost /= 3;
}
- if (values != null) {
- return new TupleBrowser(this.tree, values, direction);
+ int cost = Math.max(1, rowCost);
+ if (!ii.covering || (orderBy != null && ii.ordering == null)) {
+ cost = (int)(cost * Math.log(cost)/LN_2);
}
- return new TupleBrowser(this.tree, lower, upper, direction);
+ return cost;
}
+
+ private TupleBrowser createTupleBrower(Criteria condition, boolean direction) throws TeiidComponentException {
+ IndexInfo ii = new IndexInfo(this, null, condition, null, true);
+ ii.ordering = direction;
+ return ii.createTupleBrowser();
+ }
public int getRowCount() {
return tree.getRowCount();
@@ -411,6 +417,11 @@
public void remove() {
tree.remove();
+ if (this.indexTables != null) {
+ for (TempTable indexTable : this.indexTables.values()) {
+ indexTable.remove();
+ }
+ }
}
public List<ElementSymbol> getColumns() {
@@ -547,9 +558,24 @@
try {
lock.writeLock().lock();
if (remove) {
- return tree.remove(tuple);
+ List<?> result = tree.remove(tuple);
+ if (result == null) {
+ return null;
+ }
+ if (indexTables != null) {
+ //remove from each index table
+ /*for (TempTable index : this.indexTables.values()) {
+ index.tree
+ }*/
+ }
+ return result;
}
- return tree.insert(tuple, InsertMode.UPDATE);
+ List<?> result = tree.insert(tuple, InsertMode.UPDATE);
+ if (indexTables != null) {
+ //update each index table
+
+ }
+ return result;
} finally {
lock.writeLock().unlock();
}
@@ -567,6 +593,11 @@
void setUpdatable(boolean updatable) {
this.updatable = updatable;
+ if (this.indexTables != null) {
+ for (TempTable index : this.indexTables.values()) {
+ index.setUpdatable(updatable);
+ }
+ }
}
CacheHint getCacheHint() {
@@ -583,5 +614,18 @@
public boolean isUpdatable() {
return updatable;
}
+
+ @Override
+ public String toString() {
+ return tid.getID() + " (" + columns + ")\n"; //$NON-NLS-1$ //$NON-NLS-2$
+ }
+ Map getColumnMap() {
+ return this.columnMap;
+ }
+
+ STree getTree() {
+ return tree;
+ }
+
}
\ No newline at end of file
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTableDataManager.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTableDataManager.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTableDataManager.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -24,6 +24,7 @@
import java.util.ArrayList;
import java.util.Arrays;
+import java.util.Collection;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.Callable;
@@ -113,14 +114,8 @@
}, new SessionAwareCache<CachedResults>());
}
- /**
- * Constructor takes the "real" ProcessorDataManager that this object will be a proxy to,
- * and will pass most calls through to transparently. Only when a request is registered for
- * a temp group will this proxy do it's thing.
- * @param processorDataManager the real ProcessorDataManager that this object is a proxy to
- * @param cache
- */
- public TempTableDataManager(ProcessorDataManager processorDataManager, BufferManager bufferManager, Executor executor, SessionAwareCache<CachedResults> cache){
+ public TempTableDataManager(ProcessorDataManager processorDataManager, BufferManager bufferManager,
+ Executor executor, SessionAwareCache<CachedResults> cache){
this.processorDataManager = processorDataManager;
this.bufferManager = bufferManager;
this.executor = executor;
@@ -159,42 +154,9 @@
return result;
}
} else if (proc.getGroup().isGlobalTable()) {
- String fullName = context.getMetadata().getFullName(proc.getProcedureID());
- LinkedList<Object> vals = new LinkedList<Object>();
- for (SPParameter param : proc.getInputParameters()) {
- vals.add(((Constant)param.getExpression()).getValue());
- }
- //collapse the hash to single byte for the key to restrict the possible results to 256
- int hash = vals.hashCode();
- hash |= (hash >>> 16);
- hash |= (hash >>> 8);
- hash &= 0x000000ff;
- CacheID cid = new CacheID(new ParseInfo(), fullName + hash, context.getVdbName(),
- context.getVdbVersion(), context.getConnectionID(), context.getUserName());
- cid.setParameters(vals);
- CachedResults results = cache.get(cid);
- if (results != null) {
- TupleBuffer buffer = results.getResults();
- return buffer.createIndexedTupleSource();
- }
- CacheHint hint = proc.getCacheHint();
- proc.setCacheHint(null);
- Option option = new Option();
- option.setNoCache(true);
- option.addNoCacheGroup(fullName);
- proc.setOption(option);
- int determinismLevel = context.resetDeterminismLevel();
- QueryProcessor qp = context.getQueryProcessorFactory().createQueryProcessor(proc.toString(), fullName.toUpperCase(), context);
- qp.setNonBlocking(true);
- BatchCollector bc = qp.createBatchCollector();
- TupleBuffer tb = bc.collectTuples();
- CachedResults cr = new CachedResults();
- cr.setResults(tb);
- cr.setHint(hint);
- cache.put(cid, context.getDeterminismLevel(), cr, hint != null?hint.getTtl():null);
- context.setDeterminismLevel(determinismLevel);
- return tb.createIndexedTupleSource();
+ return handleCachedProcedure(context, proc);
}
+ return null; //it's not a stored procedure we want to handle
}
GroupSymbol group = ((ProcedureContainer)command).getGroup();
@@ -248,6 +210,50 @@
return null;
}
+ private TupleSource handleCachedProcedure(CommandContext context,
+ StoredProcedure proc) throws TeiidComponentException,
+ QueryMetadataException, TeiidProcessingException {
+ String fullName = context.getMetadata().getFullName(proc.getProcedureID());
+ LogManager.logDetail(LogConstants.CTX_DQP, "processing cached procedure request for", fullName); //$NON-NLS-1$
+ LinkedList<Object> vals = new LinkedList<Object>();
+ for (SPParameter param : proc.getInputParameters()) {
+ vals.add(((Constant)param.getExpression()).getValue());
+ }
+ //collapse the hash to single byte for the key to restrict the possible results to 256
+ int hash = vals.hashCode();
+ hash |= (hash >>> 16);
+ hash |= (hash >>> 8);
+ hash &= 0x000000ff;
+ CacheID cid = new CacheID(new ParseInfo(), fullName + hash, context.getVdbName(),
+ context.getVdbVersion(), context.getConnectionID(), context.getUserName());
+ cid.setParameters(vals);
+ CachedResults results = cache.get(cid);
+ if (results != null) {
+ TupleBuffer buffer = results.getResults();
+ return buffer.createIndexedTupleSource();
+ }
+ //construct a query with a no cache hint
+ //note that it's safe to use the stringified form of the parameters because
+ //it's not possible to use xml/clob/blob/object
+ CacheHint hint = proc.getCacheHint();
+ proc.setCacheHint(null);
+ Option option = new Option();
+ option.setNoCache(true);
+ option.addNoCacheGroup(fullName);
+ proc.setOption(option);
+ int determinismLevel = context.resetDeterminismLevel();
+ QueryProcessor qp = context.getQueryProcessorFactory().createQueryProcessor(proc.toString(), fullName.toUpperCase(), context);
+ qp.setNonBlocking(true);
+ BatchCollector bc = qp.createBatchCollector();
+ TupleBuffer tb = bc.collectTuples();
+ CachedResults cr = new CachedResults();
+ cr.setResults(tb);
+ cr.setHint(hint);
+ cache.put(cid, context.getDeterminismLevel(), cr, hint != null?hint.getTtl():null);
+ context.setDeterminismLevel(determinismLevel);
+ return tb.createIndexedTupleSource();
+ }
+
private TupleSource handleSystemProcedures(CommandContext context, StoredProcedure proc)
throws TeiidComponentException, QueryMetadataException,
QueryProcessingException, QueryResolverException,
@@ -259,6 +265,7 @@
Object groupID = validateMatView(metadata, proc);
String matViewName = metadata.getFullName(groupID);
String matTableName = RelationalPlanner.MAT_PREFIX+matViewName.toUpperCase();
+ LogManager.logDetail(LogConstants.CTX_MATVIEWS, "processing refreshmatview for", matViewName); //$NON-NLS-1$
MatTableInfo info = globalStore.getMatTableInfo(matTableName);
boolean invalidate = Boolean.TRUE.equals(((Constant)proc.getParameter(1).getExpression()).getValue());
MatState oldState = info.setState(MatState.NEEDS_LOADING, invalidate?Boolean.FALSE:null);
@@ -387,23 +394,24 @@
QueryMetadataInterface metadata = context.getMetadata();
Create create = new Create();
create.setTable(group);
- create.setColumns(ResolverUtil.resolveElementsInGroup(group, metadata));
+ List<ElementSymbol> allColumns = ResolverUtil.resolveElementsInGroup(group, metadata);
+ create.setColumns(allColumns);
Object pk = metadata.getPrimaryKey(group.getMetadataID());
if (pk != null) {
- for (Object col : metadata.getElementIDsInKey(pk)) {
- create.getPrimaryKey().add(create.getColumns().get(metadata.getPosition(col)-1));
- }
+ List<ElementSymbol> pkColumns = resolveIndex(metadata, allColumns, pk);
+ create.getPrimaryKey().addAll(pkColumns);
}
TempTable table = globalStore.addTempTable(tableName, create, bufferManager, false);
table.setUpdatable(false);
CacheHint hint = table.getCacheHint();
+ boolean updatable = false;
if (hint != null) {
table.setPreferMemory(hint.getPrefersMemory());
if (hint.getTtl() != null) {
info.setTtl(table.getCacheHint().getTtl());
}
if (pk != null) {
- table.setUpdatable(hint.isUpdatable());
+ updatable = hint.isUpdatable();
}
}
int rowCount = -1;
@@ -418,6 +426,16 @@
//TODO: if this insert fails, it's unnecessary to do the undo processing
table.insert(ts, table.getColumns());
rowCount = table.getRowCount();
+ //TODO: could pre-process indexes to remove overlap
+ for (Object index : metadata.getIndexesInGroup(group.getMetadataID())) {
+ List<ElementSymbol> columns = resolveIndex(metadata, allColumns, index);
+ table.addIndex(columns);
+ }
+ for (Object key : metadata.getUniqueKeysInGroup(group.getMetadataID())) {
+ List<ElementSymbol> columns = resolveIndex(metadata, allColumns, key);
+ table.addIndex(columns);
+ }
+ table.setUpdatable(updatable);
} catch (TeiidComponentException e) {
LogManager.logError(LogConstants.CTX_MATVIEWS, e, QueryExecPlugin.Util.getString("TempTableDataManager.failed_load", tableName)); //$NON-NLS-1$
throw e;
@@ -436,6 +454,20 @@
return rowCount;
}
+ /**
+ * Return a list of ElementSymbols for the given index/key object
+ */
+ private List<ElementSymbol> resolveIndex(QueryMetadataInterface metadata,
+ List<ElementSymbol> allColumns, Object pk)
+ throws TeiidComponentException, QueryMetadataException {
+ Collection<?> pkIds = metadata.getElementIDsInKey(pk);
+ List<ElementSymbol> pkColumns = new ArrayList<ElementSymbol>(pkIds.size());
+ for (Object col : pkIds) {
+ pkColumns.add(allColumns.get(metadata.getPosition(col)-1));
+ }
+ return pkColumns;
+ }
+
public Object lookupCodeValue(CommandContext context, String codeTableName,
String returnElementName, String keyElementName, Object keyValue)
throws BlockedException, TeiidComponentException,
Modified: branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java
===================================================================
--- branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestMaterialization.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -62,7 +62,7 @@
dataManager = new TempTableDataManager(hdm, BufferManagerFactory.getStandaloneBufferManager());
}
- private void execute(String sql, List... expectedResults) throws Exception {
+ private void execute(String sql, List<?>... expectedResults) throws Exception {
CommandContext cc = TestProcessor.createCommandContext();
cc.setTempTableStore(tempStore);
cc.setGlobalTableStore(globalStore);
@@ -114,5 +114,15 @@
execute("call sp1(null)");
assertEquals(3, hdm.getCommandHistory().size());
}
+
+ @Test public void testCoveringSecondaryIndex() throws Exception {
+ execute("SELECT * from vgroup3 where y in ('zne', 'zwo') order by y desc", Arrays.asList("two", "zwo"), Arrays.asList("one", "zne"));
+ execute("SELECT * from vgroup3 where y is null", Arrays.asList((String)null, (String)null));
+ }
+
+ @Test public void testNonCoveringSecondaryIndex() throws Exception {
+ execute("SELECT * from vgroup5 where y in ('zne', 'zwo') order by y desc", Arrays.asList("two", "zwo", 1), Arrays.asList("one", "zne", 1));
+ execute("SELECT * from vgroup5 where y is null", Arrays.asList((String)null, (String)null, 1));
+ }
}
Modified: branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java
===================================================================
--- branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java 2010-08-27 18:51:58 UTC (rev 2496)
+++ branches/7.1.x/engine/src/test/java/org/teiid/query/unittest/RealMetadataFactory.java 2010-08-27 19:00:01 UTC (rev 2497)
@@ -343,6 +343,7 @@
new String[] { "x" }, //$NON-NLS-1$
new String[] { DataTypeManager.DefaultDataTypes.STRING});
+ //covering index
QueryNode vTrans3 = new QueryNode("VGroup3", "SELECT x, 'z' || substring(x, 2) as y FROM matsrc"); //$NON-NLS-1$ //$NON-NLS-2$
Table vGroup3 = createVirtualGroup("VGroup3", virtModel, vTrans3); //$NON-NLS-1$
vGroup3.setMaterialized(true);
@@ -350,7 +351,8 @@
new String[] { "x", "y" }, //$NON-NLS-1$
new String[] { DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING});
- createKey("pk", vGroup3, vElements3.subList(0, 1));
+ createKey(KeyRecord.Type.Primary, "pk", vGroup3, vElements3.subList(0, 1));
+ createKey(KeyRecord.Type.Index, "idx", vGroup3, vElements3.subList(1, 2));
QueryNode vTrans4 = new QueryNode("VGroup4", "/*+ cache(ttl:100) */ SELECT x FROM matsrc"); //$NON-NLS-1$ //$NON-NLS-2$
Table vGroup4 = createVirtualGroup("VGroup4", virtModel, vTrans4); //$NON-NLS-1$
@@ -359,6 +361,17 @@
new String[] { "x" }, //$NON-NLS-1$
new String[] { DataTypeManager.DefaultDataTypes.STRING});
+ //non-covering index
+ QueryNode vTrans5 = new QueryNode("VGroup5", "SELECT x, 'z' || substring(x, 2) as y, 1 as z FROM matsrc"); //$NON-NLS-1$ //$NON-NLS-2$
+ Table vGroup5 = createVirtualGroup("VGroup5", virtModel, vTrans5); //$NON-NLS-1$
+ vGroup5.setMaterialized(true);
+ List<Column> vElements5 = createElements(vGroup5,
+ new String[] { "x", "y", "z" }, //$NON-NLS-1$
+ new String[] { DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.STRING, DataTypeManager.DefaultDataTypes.INTEGER});
+
+ createKey(KeyRecord.Type.Primary, "pk", vGroup5, vElements5.subList(0, 1));
+ createKey(KeyRecord.Type.Index, "idx", vGroup5, vElements5.subList(1, 2));
+
Schema sp = createVirtualModel("sp", metadataStore); //$NON-NLS-1$
ColumnSet<Procedure> rs = createResultSet("sp1.vsprs1", new String[] { "StringKey" }, new String[] { DataTypeManager.DefaultDataTypes.STRING }); //$NON-NLS-1$ //$NON-NLS-2$
ProcedureParameter param = createParameter("param1", ParameterInfo.IN, DataTypeManager.DefaultDataTypes.STRING); //$NON-NLS-1$
@@ -378,13 +391,22 @@
* metadata IDs)
* @return key metadata object
*/
- public static KeyRecord createKey(String name, Table group, List<Column> elements) {
- KeyRecord key = new KeyRecord(org.teiid.metadata.KeyRecord.Type.Primary);
+ public static KeyRecord createKey(KeyRecord.Type type, String name, Table group, List<Column> elements) {
+ KeyRecord key = new KeyRecord(type);
key.setName(name);
for (Column column : elements) {
key.addColumn(column);
}
- group.setPrimaryKey(key);
+ switch (type) {
+ case Primary:
+ group.setPrimaryKey(key);
+ break;
+ case Index:
+ group.getIndexes().add(key);
+ break;
+ default:
+ throw new AssertionError("TODO");
+ }
return key;
}
13 years, 8 months
- 1
- 0
teiid SVN: r2496 - in branches/7.1.x: client/src/main/java/org/teiid/adminapi/impl and 4 other directories.
by teiid-commits＠lists.jboss.org
Author: shawkins
Date: 2010-08-27 14:51:58 -0400 (Fri, 27 Aug 2010)
New Revision: 2496
Modified:
branches/7.1.x/client/src/main/java/org/teiid/adminapi/DataPolicy.java
branches/7.1.x/client/src/main/java/org/teiid/adminapi/impl/DataPolicyMetadata.java
branches/7.1.x/client/src/main/resources/vdb-deployer.xsd
branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml
branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/AuthorizationValidationVisitor.java
branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/DQPWorkContext.java
branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/Request.java
branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestAuthorizationValidationVisitor.java
branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java
Log:
TEIID-1231 update to change the functioning of visibility.
Modified: branches/7.1.x/client/src/main/java/org/teiid/adminapi/DataPolicy.java
===================================================================
--- branches/7.1.x/client/src/main/java/org/teiid/adminapi/DataPolicy.java 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/client/src/main/java/org/teiid/adminapi/DataPolicy.java 2010-08-27 18:51:58 UTC (rev 2496)
@@ -51,6 +51,7 @@
*/
List<String> getMappedRoleNames();
+ boolean isAnyAuthenticated();
interface DataPermission {
/**
Modified: branches/7.1.x/client/src/main/java/org/teiid/adminapi/impl/DataPolicyMetadata.java
===================================================================
--- branches/7.1.x/client/src/main/java/org/teiid/adminapi/impl/DataPolicyMetadata.java 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/client/src/main/java/org/teiid/adminapi/impl/DataPolicyMetadata.java 2010-08-27 18:51:58 UTC (rev 2496)
@@ -52,6 +52,8 @@
protected String name;
@XmlElement(name = "description")
protected String description;
+ @XmlAttribute(name = "any-authenticated")
+ protected boolean anyAuthenticated;
@XmlElement(name = "permission")
protected PermissionMap permissions = new PermissionMap(new KeyBuilder<PermissionMetaData>() {
@@ -251,4 +253,15 @@
return sb.toString();
}
}
+
+
+ @Override
+ @ManagementProperty(description="Indicates if the role is mapped to any authenticated user.")
+ public boolean isAnyAuthenticated() {
+ return false;
+ }
+
+ public void setAnyAuthenticated(boolean anyAuthenticated) {
+ this.anyAuthenticated = anyAuthenticated;
+ }
}
Modified: branches/7.1.x/client/src/main/resources/vdb-deployer.xsd
===================================================================
--- branches/7.1.x/client/src/main/resources/vdb-deployer.xsd 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/client/src/main/resources/vdb-deployer.xsd 2010-08-27 18:51:58 UTC (rev 2496)
@@ -98,6 +98,7 @@
<xs:element name="mapped-role-name" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
<xs:attribute name="name" type="xs:string" use="required"/>
+ <xs:attribute name="any-authenticated" type="xs:boolean"/>
</xs:complexType>
</xs:element>
<xs:element name="entry" minOccurs="0" maxOccurs="unbounded">
Modified: branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml
===================================================================
--- branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml 2010-08-27 18:51:58 UTC (rev 2496)
@@ -5,7 +5,7 @@
]>
<chapter id="dataroles">
<title>Data Roles</title>
- <para>Data roles, also called entitlements, are sets of permissions that are defined
+ <para>Data roles, also called entitlements, are sets of permissions defined
per VDB that dictate data access (create, read, update, delete). The use of data roles can be disabled system wide with the property in
&jboss-beans; file in bean configuration section of <code>RuntimeEngineDeployer</code> with property <code>useDataRoles</code>.</para>
@@ -26,14 +26,16 @@
Thus it is possible to set very general permissions at high-level resource path names and to override only as necessary at more specific resource paths.
</para>
- <para>Permission grants are only needed for resources that are visible.
- All non-visible, typically physical, models in a VDB are automatically
- inaccessible by user level requests. Permissions are also only applied
- to the columns and tables in the user query - not to every resource
- accessed transitively through views and procedures.
+ <para>Permission grants are only needed for resources that a role needs access to.
+ Permissions are also only applied to the columns/tables/procedures in the user query - not to every resource
+ accessed transitively through view and procedure definitions.
It is important therefore to ensure that permission grants are applied
- consistently across visible models that access the same resources.
+ consistently across models that access the same resources.
</para>
+
+ <warning><para>Unlike previous versions of Teiid, non-visible models are accessible by user queries.
+ To restrict user access at a model level, at least one data role should be created to enable data role checking.
+ In turn that role can be mapped to any authenticated user and should not grant permissions to models that should be inaccessable.</para></warning>
<orderedlist>
<para>To process a <emphasis>SELECT</emphasis> statement or a stored procedure execution, the user account requires the following access rights:</para>
@@ -66,6 +68,16 @@
</orderedlist>
</section>
+
+ <section>
+ <title>Role Mapping</title>
+ <para>Each Teiid data role can be mapped to any number of container roles or any authenticated user.
+ You may control role membership through whatever system the Teiid security domain login modules are associated with.
+ The kit includes example files for use with the UsersRolesLoginModule - see teiid-security-roles.properties.</para>
+ <para>It is possible for a user to have any number of container roles, which in turn imply a subset of Teiid data roles.
+ Each applicable Teiid data role contributes cumulatively to the permissions of the user.
+ No one role supercedes or negates the permissions of the other data roles.</para>
+ </section>
<section>
<title>XML Definition</title>
Modified: branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/AuthorizationValidationVisitor.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/AuthorizationValidationVisitor.java 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/AuthorizationValidationVisitor.java 2010-08-27 18:51:58 UTC (rev 2496)
@@ -36,8 +36,6 @@
import org.teiid.adminapi.DataPolicy;
import org.teiid.adminapi.impl.DataPolicyMetadata;
-import org.teiid.adminapi.impl.ModelMetaData;
-import org.teiid.adminapi.impl.VDBMetaData;
import org.teiid.api.exception.query.QueryMetadataException;
import org.teiid.core.TeiidComponentException;
import org.teiid.core.TeiidProcessingException;
@@ -75,32 +73,16 @@
STORED_PROCEDURE;
}
- private VDBMetaData vdb;
private HashMap<String, DataPolicy> allowedPolicies;
private String userName;
- private boolean useEntitlements;
- public AuthorizationValidationVisitor(VDBMetaData vdb, boolean useEntitlements, HashMap<String, DataPolicy> policies, String user) {
- this.vdb = vdb;
+ public AuthorizationValidationVisitor(HashMap<String, DataPolicy> policies, String user) {
this.allowedPolicies = policies;
this.userName = user;
- this.useEntitlements = useEntitlements;
}
// ############### Visitor methods for language objects ##################
- @Override
- public void visit(GroupSymbol obj) {
- try {
- Object modelID = getMetadata().getModelID(obj.getMetadataID());
- this.validateModelVisibility(modelID, obj);
- } catch(QueryMetadataException e) {
- handleException(e, obj);
- } catch(TeiidComponentException e) {
- handleException(e, obj);
- }
- }
-
public void visit(Delete obj) {
validateEntitlements(obj);
}
@@ -118,7 +100,6 @@
}
public void visit(StoredProcedure obj) {
- this.validateModelVisibility(obj.getModelID(), obj.getGroup());
validateEntitlements(obj);
}
@@ -126,7 +107,6 @@
if (FunctionLibrary.LOOKUP.equalsIgnoreCase(obj.getName())) {
try {
ResolverUtil.ResolvedLookup lookup = ResolverUtil.resolveLookup(obj, this.getMetadata());
- validateModelVisibility(getMetadata().getModelID(lookup.getGroup().getMetadataID()), lookup.getGroup());
List<Symbol> symbols = new LinkedList<Symbol>();
symbols.add(lookup.getGroup());
symbols.add(lookup.getKeyElement());
@@ -289,30 +269,10 @@
}
- protected void validateModelVisibility(Object modelID, GroupSymbol group) {
- if(modelID instanceof TempMetadataID){
- return;
- }
- try {
- String modelName = getMetadata().getFullName(modelID);
- ModelMetaData model = vdb.getModel(modelName);
- if(!model.isVisible()) {
- handleValidationError(DQPPlugin.Util.getString("ERR.018.005.0088", getMetadata().getFullName(group.getMetadataID()))); //$NON-NLS-1$
- }
- } catch (TeiidComponentException e) {
- handleException(e, group);
- }
- }
-
-
/**
* Out of resources specified, return the subset for which the specified not have authorization to access.
*/
public Set<String> getInaccessibleResources(DataPolicy.PermissionType action, Set<String> resources, Context context) {
- if (!this.useEntitlements) {
- return Collections.emptySet();
- }
-
if (LogManager.isMessageToBeRecorded(LogConstants.CTX_AUDITLOGGING, MessageLevel.DETAIL)) {
// Audit - request
AuditMessage msg = new AuditMessage(context.name(), "getInaccessibleResources-request", this.userName, resources.toArray(new String[resources.size()])); //$NON-NLS-1$
Modified: branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/DQPWorkContext.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/DQPWorkContext.java 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/DQPWorkContext.java 2010-08-27 18:51:58 UTC (rev 2496)
@@ -203,9 +203,7 @@
}
// get data roles from the VDB
- List<DataPolicy> policies = getVDB().getDataPolicies();
-
- for (DataPolicy policy : policies) {
+ for (DataPolicy policy : getVDB().getDataPolicies()) {
if (matchesPrincipal(userRoles, policy)) {
this.policies.put(policy.getName(), policy);
}
@@ -215,9 +213,14 @@
}
private boolean matchesPrincipal(Set<String> userRoles, DataPolicy policy) {
+ if (policy.isAnyAuthenticated()) {
+ return true;
+ }
List<String> roles = policy.getMappedRoleNames();
for (String role:roles) {
- return userRoles.contains(role);
+ if (userRoles.contains(role)) {
+ return true;
+ }
}
return false;
}
Modified: branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/Request.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/Request.java 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/engine/src/main/java/org/teiid/dqp/internal/process/Request.java 2010-08-27 18:51:58 UTC (rev 2496)
@@ -461,8 +461,10 @@
}
protected void validateAccess(Command command) throws QueryValidatorException, TeiidComponentException {
- AuthorizationValidationVisitor visitor = new AuthorizationValidationVisitor(this.workContext.getVDB(), this.useEntitlements, this.workContext.getAllowedDataPolicies(), this.workContext.getUserName());
- validateWithVisitor(visitor, this.metadata, command);
+ if (useEntitlements) {
+ AuthorizationValidationVisitor visitor = new AuthorizationValidationVisitor(this.workContext.getAllowedDataPolicies(), this.workContext.getUserName());
+ validateWithVisitor(visitor, this.metadata, command);
+ }
}
}
Modified: branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestAuthorizationValidationVisitor.java
===================================================================
--- branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestAuthorizationValidationVisitor.java 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestAuthorizationValidationVisitor.java 2010-08-27 18:51:58 UTC (rev 2496)
@@ -22,14 +22,16 @@
package org.teiid.dqp.internal.process;
+import static org.junit.Assert.*;
+
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Set;
-import junit.framework.TestCase;
-
+import org.junit.Ignore;
+import org.junit.Test;
import org.teiid.adminapi.DataPolicy;
import org.teiid.adminapi.DataPolicy.PermissionType;
import org.teiid.adminapi.impl.DataPolicyMetadata;
@@ -39,8 +41,6 @@
import org.teiid.api.exception.query.QueryResolverException;
import org.teiid.api.exception.query.QueryValidatorException;
import org.teiid.core.TeiidComponentException;
-import org.teiid.dqp.internal.process.AuthorizationValidationVisitor;
-import org.teiid.dqp.internal.process.Request;
import org.teiid.query.metadata.QueryMetadataInterface;
import org.teiid.query.parser.QueryParser;
import org.teiid.query.resolver.QueryResolver;
@@ -52,18 +52,10 @@
import org.teiid.query.validator.ValidatorReport;
-public class TestAuthorizationValidationVisitor extends TestCase {
+public class TestAuthorizationValidationVisitor {
public static final String CONN_ID = "connID"; //$NON-NLS-1$
- /**
- * Constructor for TestAuthorizationValidationVisitor.
- * @param name
- */
- public TestAuthorizationValidationVisitor(String name) {
- super(name);
- }
-
PermissionMetaData addResource(PermissionType type, boolean flag, String resource) {
PermissionMetaData p = new PermissionMetaData();
p.setResourceName(resource);
@@ -164,7 +156,7 @@
HashMap<String, DataPolicy> policies = new HashMap<String, DataPolicy>();
policies.put(policy.getName(), policy);
- AuthorizationValidationVisitor visitor = new AuthorizationValidationVisitor(vdb, true, policies, "test"); //$NON-NLS-1$
+ AuthorizationValidationVisitor visitor = new AuthorizationValidationVisitor(policies, "test"); //$NON-NLS-1$
ValidatorReport report = Validator.validate(command, metadata, visitor);
if(report.hasItems()) {
ValidatorFailure firstFailure = (ValidatorFailure) report.getItems().iterator().next();
@@ -184,116 +176,118 @@
}
}
- public void testEverythingAccessible() throws Exception {
+ @Test public void testEverythingAccessible() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT e1 FROM pm1.g1", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testEverythingAccessible1() throws Exception {
+ @Test public void testEverythingAccessible1() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT e1 FROM (select e1 from pm1.g1) x", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testEverythingAccessible2() throws Exception {
+ @Test public void testEverythingAccessible2() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT lookup('pm1.g1', 'e1', 'e1', '1'), e1 FROM (select e1 from pm1.g1) x", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testInaccesibleElement() throws Exception {
+ @Test public void testInaccesibleElement() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT e2 FROM pm1.g1", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g1.e2"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testInaccesibleElement2() throws Exception {
+ @Test public void testInaccesibleElement2() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT lookup('pm1.g1', 'e1', 'e2', '1')", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g1.e2"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testInaccesibleGroup() throws Exception {
+ @Test public void testInaccesibleGroup() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT e1 FROM pm1.g2", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g2", "pm1.g2.e1"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
}
- public void testInsert() throws Exception {
+ @Test public void testInsert() throws Exception {
helpTest(exampleAuthSvc1(), "INSERT INTO pm1.g1 (e1, e2, e3, e4) VALUES ('x', 5, {b'true'}, 1.0)", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testInsertInaccessible() throws Exception {
+ @Test public void testInsertInaccessible() throws Exception {
helpTest(exampleAuthSvc1(), "INSERT INTO pm1.g2 (e1, e2, e3, e4) VALUES ('x', 5, {b'true'}, 1.0)", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g2.e1"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testUpdate() throws Exception {
+ @Test public void testUpdate() throws Exception {
helpTest(exampleAuthSvc1(), "UPDATE pm1.g1 SET e2 = 5", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testUpdateCriteriaInaccessibleForRead() throws Exception {
+ @Test public void testUpdateCriteriaInaccessibleForRead() throws Exception {
helpTest(exampleAuthSvc1(), "UPDATE pm1.g2 SET e2 = 5 WHERE e1 = 'x'", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g2.e1"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testUpdateElementInaccessibleForUpdate() throws Exception {
+ @Test public void testUpdateElementInaccessibleForUpdate() throws Exception {
helpTest(exampleAuthSvc1(), "UPDATE pm1.g1 SET e1 = 5 WHERE e1 = 'x'", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g1.e1"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testDelete() throws Exception {
+ @Test public void testDelete() throws Exception {
helpTest(exampleAuthSvc1(), "DELETE FROM pm1.g1", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testDeleteCriteriaInaccesibleForRead() throws Exception {
+ @Test public void testDeleteCriteriaInaccesibleForRead() throws Exception {
helpTest(exampleAuthSvc1(), "DELETE FROM pm1.g2 WHERE e1 = 'x'", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g2.e1"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testDeleteInaccesibleGroup() throws Exception {
+ @Test public void testDeleteInaccesibleGroup() throws Exception {
helpTest(exampleAuthSvc1(), "DELETE FROM pm1.g3", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g3"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testProc() throws Exception {
+ @Test public void testProc() throws Exception {
helpTest(exampleAuthSvc1(), "EXEC pm1.sq1()", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testProcInaccesible() throws Exception {
+ @Test public void testProcInaccesible() throws Exception {
helpTest(exampleAuthSvc1(), "EXEC pm1.sq2('xyz')", FakeMetadataFactory.example1Cached(), new String[] {"pm1.sq2"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testSelectIntoEverythingAccessible() throws Exception {
+ @Test public void testSelectIntoEverythingAccessible() throws Exception {
helpTest(exampleAuthSvc2(), "SELECT e1, e2, e3, e4 INTO pm1.g2 FROM pm2.g1", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testSelectIntoTarget_e1_NotAccessible() throws Exception {
+ @Test public void testSelectIntoTarget_e1_NotAccessible() throws Exception {
helpTest(exampleAuthSvc2(), "SELECT e1, e2, e3, e4 INTO pm2.g2 FROM pm2.g1", FakeMetadataFactory.example1Cached(), new String[] {"pm2.g2.e2","pm2.g2.e4","pm2.g2.e3"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$
}
- public void testSelectIntoTarget_e1e2_NotAccessible() throws Exception {
+ @Test public void testSelectIntoTarget_e1e2_NotAccessible() throws Exception {
helpTest(exampleAuthSvc2(), "SELECT e1, e2, e3, e4 INTO pm3.g2 FROM pm2.g1", FakeMetadataFactory.example1Cached(), new String[] {"pm3.g2.e4", "pm3.g2.e3"},FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$
}
- public void testTempTableSelectInto() throws Exception {
+ @Test public void testTempTableSelectInto() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT e1 INTO #temp FROM pm1.g1", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testTempTableSelectInto1() throws Exception {
+ @Test public void testTempTableSelectInto1() throws Exception {
helpTest(exampleAuthSvc1(), "SELECT e1, e2 INTO #temp FROM pm1.g1", FakeMetadataFactory.example1Cached(), new String[] {"pm1.g1.e2"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testTempTableInsert() throws Exception {
+ @Test public void testTempTableInsert() throws Exception {
helpTest(exampleAuthSvc2(), "insert into #temp (e1, e2, e3, e4) values ('1', '2', '3', '4')", FakeMetadataFactory.example1Cached(), new String[] {}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$
}
- public void testXMLAccessible() throws Exception {
+ @Test public void testXMLAccessible() throws Exception {
helpTest(exampleAuthSvc2(), "select * from xmltest.doc1", FakeMetadataFactory.example1Cached(), new String[] {"xmltest.doc1"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
- public void testXMLInAccessible() throws Exception {
+ @Test public void testXMLInAccessible() throws Exception {
helpTest(exampleAuthSvc1(), "select * from xmltest.doc1", FakeMetadataFactory.example1Cached(), new String[] {"xmltest.doc1"}, FakeMetadataFactory.example1VDB()); //$NON-NLS-1$ //$NON-NLS-2$
}
private void helpTestLookupVisibility(boolean visible) throws QueryParserException, QueryValidatorException, TeiidComponentException {
VDBMetaData vdb = FakeMetadataFactory.example1VDB();
vdb.getModel("pm1").setVisible(visible); //$NON-NLS-1$
- AuthorizationValidationVisitor mvvv = new AuthorizationValidationVisitor(vdb, false, new HashMap<String, DataPolicy>(), "test"); //$NON-NLS-1$
+ AuthorizationValidationVisitor mvvv = new AuthorizationValidationVisitor(new HashMap<String, DataPolicy>(), "test"); //$NON-NLS-1$
String sql = "select lookup('pm1.g1', 'e1', 'e2', 1)"; //$NON-NLS-1$
Command command = QueryParser.getQueryParser().parseCommand(sql);
Request.validateWithVisitor(mvvv, FakeMetadataFactory.example1Cached(), command);
}
- public void testLookupVisibility() throws Exception {
+ @Ignore("visibility no longer ristricts access")
+ @Test public void testLookupVisibility() throws Exception {
helpTestLookupVisibility(true);
}
- public void testLookupVisibilityFails() throws Exception {
+ @Ignore("visibility no longer ristricts access")
+ @Test public void testLookupVisibilityFails() throws Exception {
try {
helpTestLookupVisibility(false);
fail("expected exception"); //$NON-NLS-1$
Modified: branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java
===================================================================
--- branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java 2010-08-26 19:18:29 UTC (rev 2495)
+++ branches/7.1.x/engine/src/test/java/org/teiid/dqp/internal/process/TestDQPCore.java 2010-08-27 18:51:58 UTC (rev 2496)
@@ -31,6 +31,7 @@
import org.junit.After;
import org.junit.Before;
+import org.junit.Ignore;
import org.junit.Test;
import org.mockito.Mockito;
import org.teiid.api.exception.query.QueryResolverException;
@@ -172,6 +173,7 @@
}
}
+ @Ignore("visibility no longer ristricts access")
@Test public void testLookupVisibility() throws Exception {
helpTestVisibilityFails("select lookup('bqt3.smalla', 'intkey', 'stringkey', '?')"); //$NON-NLS-1$
}
13 years, 8 months
- 1
- 0
teiid SVN: r2495 - branches/7.1.x/documentation/reference/src/main/docbook/en-US/content.
by teiid-commits＠lists.jboss.org
Author: shawkins
Date: 2010-08-26 15:18:29 -0400 (Thu, 26 Aug 2010)
New Revision: 2495
Modified:
branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml
Log:
TEIID-1220 changing data role checking to be enabled by default, but to only check against vdbs with data roles
Modified: branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml
===================================================================
--- branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml 2010-08-26 18:24:46 UTC (rev 2494)
+++ branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/dataroles.xml 2010-08-26 19:18:29 UTC (rev 2495)
@@ -6,14 +6,13 @@
<chapter id="dataroles">
<title>Data Roles</title>
<para>Data roles, also called entitlements, are sets of permissions that are defined
- per VDB that dictate data access (create, read, update, delete). The use of data roles is controlled system wide with the property in
- <code><jboss-install>/server/<profile>/deploy/teiid/teiid-jboss-beans.xml</code> file
- in bean configuration section of <code>RuntimeEngineDeployer</code> with property <code>useDataRoles</code>.</para>
+ per VDB that dictate data access (create, read, update, delete). The use of data roles can be disabled system wide with the property in
+ &jboss-beans; file in bean configuration section of <code>RuntimeEngineDeployer</code> with property <code>useDataRoles</code>.</para>
- <para>Once data roles are enabled, the access permissions defined in a VDB will be enforced by the Teiid Server.
- </para>
+ <para>If data roles is enabled and data roles are defined in a VDB, then access permissions will be enforced by the Teiid Server.</para>
- <warning><para>Teiid uses a deny by default permission system, so all VDBs deployed to the server will need roles granting access with this feature enabled.</para></warning>
+ <warning><para>Unlike previous versions of Teiid data roles will only be checked if present in a VDB.
+ A VDB deployed without data roles is open for use by any authenticated user.</para></warning>
<section>
<title>Permissions</title>
13 years, 8 months
- 1
- 0
teiid SVN: r2494 - in branches/7.1.x/engine/src: main/java/org/teiid/query/tempdata and 1 other directories.
by teiid-commits＠lists.jboss.org
Author: shawkins
Date: 2010-08-26 14:24:46 -0400 (Thu, 26 Aug 2010)
New Revision: 2494
Modified:
branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java
branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestTempTables.java
Log:
TEIID-1229 fix for partial composite index handling
Modified: branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java 2010-08-26 16:01:31 UTC (rev 2493)
+++ branches/7.1.x/engine/src/main/java/org/teiid/common/buffer/TupleBrowser.java 2010-08-26 18:24:46 UTC (rev 2494)
@@ -22,6 +22,7 @@
package org.teiid.common.buffer;
+import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;
@@ -48,6 +49,8 @@
private TupleBatch values;
private boolean updated;
private boolean direction;
+
+ private boolean inPartial;
/**
* Construct a value based browser
@@ -73,10 +76,17 @@
this.tree = sTree;
this.direction = direction;
+ init(lowerBound, upperBound);
+ }
+
+ private void init(List<Object> lowerBound,
+ List<Object> upperBound)
+ throws TeiidComponentException {
if (lowerBound != null) {
+ lowerBound.addAll(Collections.nCopies(tree.getKeyLength() - lowerBound.size(), null));
setPage(lowerBound);
} else {
- page = sTree.header[0];
+ page = tree.header[0];
}
boolean valid = true;
@@ -91,7 +101,9 @@
bound = upper.page;
boundIndex = upper.index;
if (boundIndex < 0) {
- boundIndex = Math.min(upper.values.getTuples().size() - 1, -boundIndex -1);
+ //we are guaranteed by find to not get back the -1 index, unless
+ //there are now tuples, in which case a bound of -1 is fine
+ boundIndex = Math.min(upper.values.getTuples().size(), -boundIndex -1) - 1;
}
if (!direction) {
values = upper.values;
@@ -143,12 +155,17 @@
TeiidProcessingException {
for (;;) {
//first check for value iteration
- if (valueSet != null) {
+ if (!inPartial && valueSet != null) {
if (valueSet.isEmpty()) {
resetState();
return null;
}
- List<?> newValue = direction?valueSet.remove(0):valueSet.remove(valueSet.size() -1);
+ List<Object> newValue = direction?valueSet.remove(0):valueSet.remove(valueSet.size() -1);
+ if (newValue.size() < tree.getKeyLength()) {
+ init(new ArrayList<Object>(newValue), newValue);
+ inPartial = true;
+ continue;
+ }
if (values != null) {
int possibleIndex = Collections.binarySearch(values.getTuples(), newValue, tree.comparator);
if (possibleIndex >= 0) {
@@ -179,6 +196,10 @@
return values.getTuples().get(index);
}
if (page == null) {
+ if (inPartial) {
+ inPartial = false;
+ continue;
+ }
return null;
}
if (values == null) {
@@ -217,7 +238,7 @@
}
private int getOffset() {
- if (valueSet != null) {
+ if (!inPartial && valueSet != null) {
return 0;
}
return direction?1:-1;
Modified: branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java
===================================================================
--- branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-26 16:01:31 UTC (rev 2493)
+++ branches/7.1.x/engine/src/main/java/org/teiid/query/tempdata/TempTable.java 2010-08-26 18:24:46 UTC (rev 2494)
@@ -23,7 +23,6 @@
package org.teiid.query.tempdata;
import java.util.ArrayList;
-import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
@@ -123,7 +122,7 @@
private TupleBrowser browser;
private QueryTupleSource(TupleBrowser browser, Map map,
- List<SingleElementSymbol> projectedCols, Criteria condition) {
+ List<? extends SingleElementSymbol> projectedCols, Criteria condition) {
this.browser = browser;
this.indexes = RelationalNode.getProjectionIndexes(map, projectedCols);
this.eval = new Evaluator(map, null, null);
@@ -281,47 +280,46 @@
this.leafBatchSize = bm.getSchemaSize(columns.subList(0, primaryKeyLength));
}
+
private int reserveBuffers() {
return bm.reserveBuffers(leafBatchSize + (tree.getHeight() - 1)*keyBatchSize, BufferReserveMode.WAIT);
}
- public TupleSource createTupleSource(final List<SingleElementSymbol> projectedCols, final Criteria condition, OrderBy orderBy) throws TeiidComponentException, TeiidProcessingException {
+ public TupleSource createTupleSource(final List<? extends SingleElementSymbol> projectedCols, final Criteria condition, OrderBy orderBy) throws TeiidComponentException, TeiidProcessingException {
Map map = RelationalNode.createLookupMap(getColumns());
Boolean direction = null;
boolean orderByUsingIndex = false;
if (orderBy != null && rowId == null) {
int[] orderByIndexes = RelationalNode.getProjectionIndexes(map, orderBy.getSortKeys());
- if (orderByIndexes.length < tree.getKeyLength()) {
- orderByUsingIndex = false;
- } else {
- orderByUsingIndex = true;
- for (int i = 0; i < tree.getKeyLength(); i++) {
- if (orderByIndexes[i] != i) {
+ orderByUsingIndex = true;
+ for (int i = 0; i < tree.getKeyLength(); i++) {
+ if (orderByIndexes.length <= i) {
+ break;
+ }
+ if (orderByIndexes[i] != i) {
+ orderByUsingIndex = false;
+ break;
+ }
+ }
+ if (orderByUsingIndex) {
+ for (OrderByItem item : orderBy.getOrderByItems()) {
+ if (item.getNullOrdering() != null) {
orderByUsingIndex = false;
break;
}
- }
- if (orderByUsingIndex) {
- for (int i = 0; i < tree.getKeyLength(); i++) {
- OrderByItem item = orderBy.getOrderByItems().get(i);
- if (item.getNullOrdering() != null) {
+ if (item.isAscending()) {
+ if (direction == null) {
+ direction = OrderBy.ASC;
+ } else if (direction != OrderBy.ASC) {
orderByUsingIndex = false;
break;
}
- if (item.isAscending()) {
- if (direction == null) {
- direction = OrderBy.ASC;
- } else if (direction != OrderBy.ASC) {
- orderByUsingIndex = false;
- break;
- }
- } else if (direction == null) {
- direction = OrderBy.DESC;
- } else if (direction != OrderBy.DESC) {
- orderByUsingIndex = false;
- break;
- }
+ } else if (direction == null) {
+ direction = OrderBy.DESC;
+ } else if (direction != OrderBy.DESC) {
+ orderByUsingIndex = false;
+ break;
}
}
}
@@ -391,24 +389,11 @@
value.add(constant.getValue());
values.add(value);
}
- } else if (values != null && values.size() == 1 && indexCondition.valueSet.size() == 1) {
+ } else if (values != null && values.size() == 1 && values.iterator().next().size() == i && indexCondition.valueSet.size() == 1) {
values.iterator().next().add(indexCondition.valueSet.first().getValue());
}
}
}
- if (indexConditions.length > 0) {
- if (values != null) {
- List<Object> value = values.iterator().next();
- if (value.size() != tree.getKeyLength()) {
- values = null;
- lower = new ArrayList<Object>(value);
- lower.addAll(Collections.nCopies(tree.getKeyLength() - value.size(), null));
- upper = new ArrayList<Object>(value);
- }
- } else if (lower != null) {
- lower.addAll(Collections.nCopies(tree.getKeyLength() - lower.size(), null));
- }
- }
}
if (values != null) {
return new TupleBrowser(this.tree, values, direction);
Modified: branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestTempTables.java
===================================================================
--- branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestTempTables.java 2010-08-26 16:01:31 UTC (rev 2493)
+++ branches/7.1.x/engine/src/test/java/org/teiid/query/processor/TestTempTables.java 2010-08-26 18:24:46 UTC (rev 2494)
@@ -236,6 +236,16 @@
sampleTable();
execute("select * from x where e1 >= 'b' order by e1 desc, e2 desc", new List[] {Arrays.asList("c", 1), Arrays.asList("b", 3), Arrays.asList("b", 2)}); //$NON-NLS-1$
}
+
+ @Test public void testCompositeKeyPartial5() throws Exception {
+ sampleTable();
+ execute("select * from x where e1 in ('a', 'b')", new List[] {Arrays.asList("a", 1), Arrays.asList("b", 2), Arrays.asList("b", 3)}); //$NON-NLS-1$
+ }
+
+ @Test public void testCompositeKeyPartial6() throws Exception {
+ execute("create local temporary table x (e1 string, e2 integer, primary key (e1, e2))", new List[] {Arrays.asList(0)}); //$NON-NLS-1$
+ execute("select * from x where e1 in ('a', 'b') order by e1 desc", new List[0]); //$NON-NLS-1$
+ }
private void sampleTable() throws Exception {
execute("create local temporary table x (e1 string, e2 integer, primary key (e1, e2))", new List[] {Arrays.asList(0)}); //$NON-NLS-1$
13 years, 8 months
- 1
- 0