[teiid-commits] teiid SVN: r2467 - in trunk: build/assembly/jboss-container and 6 other directories.

Author: shawkins Date: 2010-08-17 09:42:24 -0400 (Tue, 17 Aug 2010) New Revision: 2467 Modified: trunk/build/assembly/docs.xml trunk/build/assembly/jboss-container/dist.xml trunk/documentation/caching-guide/ trunk/documentation/caching-guide/src/main/docbook/en-US/content/cachehint.xml trunk/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml trunk/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml trunk/documentation/caching-guide/src/main/docbook/en-US/content/overview.xml trunk/documentation/caching-guide/src/main/docbook/en-US/content/resultset.xml trunk/documentation/caching-guide/src/main/docbook/en-US/main.xml trunk/documentation/reference/src/main/docbook/en-US/content/system_schema.xml trunk/engine/src/main/java/org/teiid/common/buffer/STree.java trunk/engine/src/test/java/org/teiid/cache/TestDefaultCache.java Log: TEIID-168 updating the matview/cache hint section of the docs, improving the ordered insert into a temptable, adding the caching doc to the kit Modified: trunk/build/assembly/docs.xml =================================================================== --- trunk/build/assembly/docs.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/build/assembly/docs.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -19,6 +19,7 @@ <include>org.jboss.teiid.documentation:developer-guide</include> <include>org.jboss.teiid.documentation:quick-start-example</include> <include>org.jboss.teiid.documentation:client-developers-guide</include> + <include>org.jboss.teiid.documentation:caching-guide</include> </includes> <sources> Modified: trunk/build/assembly/jboss-container/dist.xml =================================================================== --- trunk/build/assembly/jboss-container/dist.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/build/assembly/jboss-container/dist.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -90,7 +90,12 @@ <source>target/distribution/teiid-${version}-docs/client-developers-guide/en-US/pdf/teiid_client_developers_guide.pdf</source> <outputDirectory>teiid-docs</outputDirectory> <fileMode>0644</fileMode> - </file> + </file> + <file> + <source>target/distribution/teiid-${version}-docs/caching-guide/en-US/pdf/teiid_caching_guide.pdf</source> + <outputDirectory>teiid-docs</outputDirectory> + <fileMode>0644</fileMode> + </file> </files> Property changes on: trunk/documentation/caching-guide ___________________________________________________________________ Name: svn:ignore - target + target .project .classpath Legal_Notice.html .settings Modified: trunk/documentation/caching-guide/src/main/docbook/en-US/content/cachehint.xml =================================================================== --- trunk/documentation/caching-guide/src/main/docbook/en-US/content/cachehint.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/documentation/caching-guide/src/main/docbook/en-US/content/cachehint.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -3,6 +3,77 @@ %CustomDTD; ]> <chapter id="cache-hint"> - <title>Cache Hint</title> - <para></para> + <title>Cache Hint</title> + <para>A cache hint can be used to:</para> + <itemizedlist> + <listitem><para>Indicate that a user query is eligible for result set caching. + </para></listitem> + <listitem><para>Set the result set query cache entry memory preference or time to live. + </para></listitem> + <listitem><para>Set the materialized view memory preference, time to live, or updatablity. + </para></listitem> + </itemizedlist> + <para><synopsis>/*+ cache[([pref_mem] [ttl:n] [updatable])] */</synopsis> + <itemizedlist> + <listitem><para><emphasis>pref_mem</emphasis> - if present indicates that the cached results should prefer to remain in memory. They are not however required to be memory only. + </para></listitem> + <listitem><para><emphasis>ttl:n</emphasis> - if present n indicates the time to live value in milliseconds. + </para></listitem> + <listitem><para><emphasis>updatable</emphasis> - if present indicates that the cached results can be updated. + </para></listitem> + </itemizedlist> + </para> + <section> + <title>ResultSet Cache Query</title> + <para>The most basic form of the cache hint, <code>/*+ cache */</code>, is sufficient to inform the engine that the results of the non-update command should be cached.</para> + <example> + <title>PreparedStatement ResultSet Caching</title> + <programlisting>... +PreparedStatement ps = connection.prepareStatement("/*+ cache */ select col from t where col2 = ?"); +ps.setInt(1, 5); +ps.execute(); +...</programlisting> + <para>While no options are specified with the cache hint, it still informs the engine to use ResultSet caching.</para> + </example> + <para>The pref_mem and ttl options may also be used for ResultSet cache queries, however updatable only has an effect on materialized view tables.</para> + <example> + <title>Advanced ResultSet Caching</title> + <programlisting>/*+ cache(pref_mem ttl:60000 */ select col from t</programlisting> + <para>In this example the memory preference has been enabled and the time to live is set to 60000 milliseconds or 1 minute. + The ttl for an entry is actually treated as it's maximum age and the entry may be purged sooner if the maximum number of cache entries has been reached.</para> + </example> + <para>See the <link linkend="resultset">ResultSet Caching Chapter</link> for more.</para> + </section> + <section> + <title>Materialized Views</title> + <para>The cache hint, when used in the context of an internal materialized view transformation query, provides the ability to fine tune the materializated table. + The hint is not used for materialization targeted at an external source. See the <link linkend="resultset">Materialized View Chapter</link> for more on materialized views.</para> + <para>The pref_mem option also applies to internal materialized views. Internal table index pages already have a memory preference, so the perf_mem option indicates that the data pages should prefer memory as well.</para> + <section> + <title>TTL Snapshot Refresh</title> + <para>When the ttl is specified in the cache hint, a full refresh of the materialized view will be triggered automatically after the specified time interval. + The refresh is equivalent to <code>CALL SYS.refreshMatView('view name', false)</code>, but performed asynchronously so that user queries do not block on the load. + </para> + <para>The automatic loading is not intended for complex loading scenarios, as nested materialized views will be used by the refresh query.</para> + </section> + <section> + <title>Updatable</title> + <para>When the updatable option is specified, the materialized view may be targeted by the system function <code>refreshMatViewRow</code>. + The refreshMatViewRow function updates a single row of an internal materialized with the supplied key value. + The refresh query does use nested caches, so this refresh method should be used with caution. + </para> + <para>When the updatable option is not specified, accessing the materialized view table is more efficient because modifications do not need to be considered. + Therefore, only specify the updatable option if row based incremental updates are needed. Even when performing row updates, full snapshot refreshes may be needed to ensure consistency.</para> + </section> + </section> + <section> + <title>Limitations</title> + <itemizedlist> + <listitem><para>The form of the query hint must be matched exactly for the hint to have affect. + For a user query if the hint is not specified correctly, e.g. /*+ cach(pref_mem) */, it will not be used by the engine nor will + there be an informational log. As a workaround, the query plan may be checked though (see the Client Developers Guide) to see if the user command + in the plan has retained the proper hint. + </para></listitem> + </itemizedlist> + </section> </chapter> \ No newline at end of file Modified: trunk/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml =================================================================== --- trunk/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/documentation/caching-guide/src/main/docbook/en-US/content/codetable.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -34,7 +34,7 @@ </section> <section> <title>Limitations</title> - <orderedlist> + <itemizedlist> <listitem> <para>The use of the lookup function automatically performs caching; there is no option to use the lookup function and not perform @@ -43,7 +43,7 @@ <listitem> <para>No mechanism is provided to refresh code tables.</para> </listitem> - </orderedlist> + </itemizedlist> </section> <section> <title>Materialized View Alternative</title> Modified: trunk/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml =================================================================== --- trunk/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/documentation/caching-guide/src/main/docbook/en-US/content/matviews.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -4,63 +4,53 @@ ]> <chapter id="matviews"> <title>Materialized Views</title> - <para>Teiid supports Materialized Views.  These are Relational - virtual tables and views (‘virtual groups’) for which the - transformations are pre-computed and the results are stored in an - external database. When queries are issued against these virtual - groups through the Teiid Server, the cached results are used. + <para>Teiid supports materialized views.  Materialized views are just like other views, but their + transformations are pre-computed and stored just like a regular table. + When queries are issued against the views through the Teiid Server, the cached results are used.  This saves the cost of accessing all the underlying data sources and - re-computing the virtual group transforms each time a query is - executed against the group.    </para> - <para>This strategy is appropriate when the underlying data does not + re-computing the view transforms each time a query is + executed.</para> + <para>Materialized views are appropriate when the underlying data does not change rapidly, or when it is acceptable to retrieve data that is - “stale” within some period of time, or when it is preferred for + "stale" within some period of time, or when it is preferred for end-user queries to access staged data rather than placing additional - query load on operational sources.  Teiid provides a utility to - refresh materialized tables.  This utility uses the Teiid batched - update functionality.</para> + query load on operational sources.</para> <section> <title>Support Summary</title> - <orderedlist> + <itemizedlist> <listitem> <para>Caching of relational table or view records (pre-computing all transformations)</para> </listitem> <listitem> - <para>Model-based definition of virtual groups to cache</para> + <para>Model-based definition of virtual groups to cache (requires Teiid Designer)</para> </listitem> <listitem> <para>User ability to override use of materialized view cache for - specific queries</para> + specific queries through OPTION NO CACHE</para> </listitem> - <listitem> - <para>Administrative utility to initially load and refresh cached - data</para> - </listitem> - </orderedlist> + </itemizedlist> </section> <section> <title>User Interaction</title> <para>When client applications issue queries against a Relational table or view that has been defined as a materialized view, the Teiid query engine automatically routes that query to obtain the - results from the cache database.  </para> + results from the cache database.</para> <para>Individual queries may override the use of materialized views by - specifying OPTION NOCACHE on the query.  This parameter must specify + specifying OPTION NOCACHE on the query.  This parameter may specify one or more virtual groups to override (separated by commas, spaces - optional).  If no virtual groups are specified, then it is the same - as if the override option is not specified. Note that only virtual - groups specified in the user or transformation query (in the FROM - clause) can be specified to be overridden.  If there are materialized - virtual groups below that level (referenced in transformations), then - the materialized view for those tables will be used.   </para> - <para>Examples:</para> - <para>SELECT * from vg1, vg2, vg3 WHERE … OPTION NOCACHE vg1, vg3 + optional).  If no virtual groups are specified, materialized views tables will not be used transitively. </para> - <para>SELECT * from vg1, vg2, vg3 WHERE … OPTION NOCACHE</para> - <para>The second query is equivalent to:</para> - <para>SELECT * from vg1, vg2, vg3 WHERE OPTION NOCACHE vg1, vg2, vg3 - </para> + <example> + <title>Full NOCACHE</title> + <programlisting>SELECT * from vg1, vg2, vg3 WHERE … OPTION NOCACHE</programlisting> + </example> + <example> + <title>Specific NOCACHE</title> + <programlisting>SELECT * from vg1, vg2, vg3 WHERE … OPTION NOCACHE vg1, vg3</programlisting> + <para>Only the vg1 and vg3 caches will be skipped vg2 or any materialized views nested under vg1 and vg3 will be used.</para> + </example> <para>The materialization override option may be specified in virtual group transformation definitions.  In that way, transformations can specify to always use real-time data obtained directly from a source. @@ -68,359 +58,36 @@ definitions, just as with user queries.</para> </section> <section> - <title>Cache Configuration</title> - <para>Materialized View Definition</para> - <para>Materialized views are defined in the MetaBase Modeler by - setting the materialized property on a table or view in a relational + <title>Materialized View Definition</title> + <para>Materialized views are defined in Teiid Designer by + setting the materialized property on a table or view in a virtual (view) relational model.  Setting this property's value to true (the default is false) allows the data generated for this virtual table to be treated as a materialized view. </para> - <para>The Name In Source property for this table determines the name - of the physical Materialized Views tables. If left blank the default - naming scheme of MV10000001 is used, otherwise the name provided is - used.</para> - <para /> - <para>Virtual Database Definition</para> - <para>Materialized views are relational tables (or views) in one or - more models that have their materialized property set to true.  When - a set of virtual relational models containing materialized views is - bundled into a virtual database (VDB) in the MetaBase Modeler, a - physical relational model is automatically created and put into that - VDB to represent the physical cache.  The physical model is given the - well-known name Materialization.  The physical cache model will - contain one physical table for each virtual group marked as a - materialized view.  </para> - <para>Note that if no virtual groups are marked as materialized views, - then the Materialization physical cache model will not be created or - included in the VDB.</para> - <para>Virtual Database Deployment        </para> - <para>When deploying a virtual database (VDB) in the Teiid - Console, the Teiid administrator must define connector bindings - for all physical models in the VDB.  This is true for the well-known - materialization cache model also.  In the New VDB or New VDB Version - wizard, the materialization model will show up in the Connector - Binding definition panel.  The administrator should select the - appropriate JDBC connector for the data source where the materialized - view cache will reside.  </para> - <para>When deploying a VDB containing materialized views, the - Teiid administrator will be presented with the option to save - administration scripts to the local file system.  This is in the New - VDB, New VDB Version, and Import VDB wizards.  The administrator - should save the scripts to a location where they can be accessed by a - database administrator responsible for creating, loading, and - refreshing the cache database.  See Cache Administration section. - </para> - <para>This is a summary of what an administrator must do when - deploying a VDB containing materialized views:</para> - <orderedlist> - <listitem> - <para>Define connector binding from the materialization model to the - physical cache database.</para> + <note><para>It is important to ensure that all key/index information is present as these will be used by the materialization process to enhance the performance of the materialized table.</para></note> + <para>The target materialized table may also be set in the properties. If the value is left blank, the default, then internal materialization will be used. + Otherwise for external materialization, the value should reference the fully qualified name of a table (or possibly view) with the same columns as the materialized view. + The choice between internal and external materialization should be made based upon: + <itemizedlist> + <listitem><para>Does the cached data need to be fully persistent? If yes, then external materialization should be used. + Internal materialization should not survive a cluster restart.</para> </listitem> - </orderedlist> - <orderedlist> - <listitem> - <para>This defines the DBMS type, subsequently used to save cache - creation and loading scripts appropriate for that database type - </para> + <listitem><para>Is full control needed of loading and refresh? If yes, then external materialzation should be used. + Internal materialization does offer several system supported methods for refreshing, but does not give full access to the materialized table.</para> </listitem> - <listitem> - <para>The username and password used in the connector binding must - have privileges for the cache database </para> - </listitem> - <listitem> - <para>The cache database may not exist at this point, in which case - the connector binding cannot be started, and the VDB cannot be - activated.</para> - </listitem> - </orderedlist> - <orderedlist> - <listitem> - <para>Define username and password for creating and manipulating - (swapping and truncating) cache tables in the cache database. - </para> - </listitem> - </orderedlist> - <orderedlist> - <listitem> - <para>The username and password must directly connect to a schema or - catalog in the database that is appropriate for the VDB version. -  This cache may be used by multiple VDB versions, but this is in - general not recommended.</para> - </listitem> - <listitem> - <para>This username and password will be used for executing the - Create, Truncate, and Swap scripts.</para> - </listitem> - </orderedlist> - <orderedlist> - <listitem> - <para>Define username and password for connecting to Teiid, to - issue insert commands against the virtual groups during loading and - reloading.</para> - </listitem> - </orderedlist> - <orderedlist> - <listitem> - <para>This user information is not necessarily the same as the user - information used in the connector binding, although it can be. - </para> - </listitem> - </orderedlist> - <orderedlist> - <listitem> - <para>Save DDL scripts to the file system.  </para> - </listitem> - </orderedlist> - <orderedlist> - <listitem> - <para>The administrator specifies the location to save the files. - </para> - </listitem> - <listitem> - <para>The location should be accessible to the DBA or the scheduling - process that will execute the scripts against the database</para> - </listitem> - <listitem> - <para>Four scripts will be saved with names appropriate to the VDB - and version.</para> - </listitem> - </orderedlist> - <para>Cache Characteristics</para> - <para>Cache Persistence Mechanism</para> - <para>Materialized views cache their data in an external database - system.  This database may be the same as the database used as the - Teiid repository, or it may be a different one.   In general it - is recommended that the Teiid repository not be used for - materialized view caching, as heavy use could impact the performance - of other core functionality of the Teiid Server.  </para> - <para>The following DBMS systems are supported for this purpose: + </itemizedlist> </para> - <orderedlist> - <listitem> - <para>Oracle 8i or 9i or 10g</para> - </listitem> - <listitem> - <para>SQL Server 2000</para> - </listitem> - <listitem> - <para>DB2 8</para> - </listitem> - </orderedlist> - <para>Cache Operational Policy</para> + + </section> + <section> + <title>External Materialization</title> + <para>External materialized views cache their data in an external database + system. External materialized views give the administrator full control over the loading and refresh strategies.</para> <para>Since the actual physical cache for materialized views is maintained external to the Teiid system, there is no pre-defined policy for clearing and managing the cache.  These policies will be defined and enforced by administrators of the Teiid system. </para> - <para>Cache Administration</para> - <para>The cache used by materialized views is administered external to - Teiid.  This provides a great deal of flexibility on how that - cache is managed, refreshed, backed up, and otherwise administered - along with other enterprise sources.</para> - <para>Administrator Responsibilities</para> - <para>These are the responsibilities of administrators with respect to - Teiid’s materialized view data caching.</para> - <informaltable frame="all"> - <tgroup cols="3"> - <colspec colnum="1" colname="c1" /> - <colspec colnum="2" colname="c2" /> - <colspec colnum="3" colname="c3" /> - <tbody> - <row> - <entry> - <para>Operation</para> - </entry> - <entry> - <para>Run Against</para> - </entry> - <entry> - <para>Required Rights for Cache Database</para> - </entry> - <entry> - <para>Required Rights for Teiid</para> - </entry> - <entry> - <para>Scripts Used</para> - </entry> - <entry> - <para>Frequency</para> - </entry> - </row> - <row> - <entry> - <para>Create cache database</para> - </entry> - <entry> - <para>Cache database</para> - </entry> - <entry> - <para>Create</para> - </entry> - <entry> - <para>None</para> - </entry> - <entry> - <para>Create</para> - </entry> - <entry> - <para>Once per VDB version</para> - </entry> - </row> - <row> - <entry> - <para>Initially populate cache </para> - </entry> - <entry> - <para>Cache database and Teiid</para> - </entry> - <entry> - <para>Alter, Insert</para> - </entry> - <entry> - <para>CRUD</para> - </entry> - <entry> - <para>Truncate</para> - <para>Load</para> - <para>Swap</para> - </entry> - <entry> - <para>Once per VDB version</para> - </entry> - </row> - <row> - <entry> - <para>Refresh cache </para> - </entry> - <entry> - <para>Cache database and Teiid</para> - </entry> - <entry> - <para>Alter, Insert </para> - </entry> - <entry> - <para>CRUD </para> - </entry> - <entry> - <para>Truncate</para> - <para>Load</para> - <para>Swap</para> - </entry> - <entry> - <para>Desired cache refresh rate</para> - </entry> - </row> - </tbody> - </tgroup> - </informaltable> - <para /> - <para>The refresh cache operation is recurring, basically at whatever - frequency is appropriate for maintaining the desired data “freshness” - for the VDB version.  This operation can be scheduled to execute the - Teiid scripts, using standard scheduling mechanisms such as cron - on UNIX or the Windows scheduler.</para> - <para>The truncate and swap scripts are used in the initial load.  The - truncate is a no-op in this case, and the swap happens because data - is always loaded to the staging table, which must then be swapped - with the real cache table.  </para> - <para>Teiid Administrative Scripts</para> - <para>To perform the administrative operations, Teiid provides a - set of 4 scripts.  Here is a summary of the scripts.  All the scripts - use a similar template for their file names: -  VDBName_VDBVersion_Name.ddl).</para> - <informaltable frame="all"> - <tgroup cols="5"> - <tbody> - <row> - <entry> - <para>Script Base Name</para> - </entry> - <entry> - <para>Use</para> - </entry> - <entry> - <para>Description</para> - </entry> - <entry> - <para>Run Against</para> - </entry> - <entry> - <para>User Info</para> - </entry> - </row> - <row> - <entry> - <para>Create</para> - </entry> - <entry> - <para>Creation</para> - </entry> - <entry> - <para>Create tables for cache</para> - </entry> - <entry> - <para>Cache database</para> - </entry> - <entry> - <para>Script user</para> - </entry> - </row> - <row> - <entry> - <para>Truncate</para> - </entry> - <entry> - <para>Reload</para> - </entry> - <entry> - <para>Truncate temporary table when inserting new records</para> - </entry> - <entry> - <para>Cache database</para> - </entry> - <entry> - <para>Script user</para> - </entry> - </row> - <row> - <entry> - <para>Load</para> - </entry> - <entry> - <para>Load or reload </para> - </entry> - <entry> - <para>Execute queries against materialized virtual groups, insert - results into cache</para> - </entry> - <entry> - <para>Teiid</para> - </entry> - <entry> - <para>Access user</para> - </entry> - </row> - <row> - <entry> - <para>Swap</para> - </entry> - <entry> - <para>Reload</para> - </entry> - <entry> - <para>Swap temporary table with cache table after new records - inserted</para> - </entry> - <entry> - <para>Cache database</para> - </entry> - <entry> - <para>Script user</para> - </entry> - </row> - </tbody> - </tgroup> - </informaltable> + <para></para> </section> </chapter> \ No newline at end of file Modified: trunk/documentation/caching-guide/src/main/docbook/en-US/content/overview.xml =================================================================== --- trunk/documentation/caching-guide/src/main/docbook/en-US/content/overview.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/documentation/caching-guide/src/main/docbook/en-US/content/overview.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -6,12 +6,8 @@ <title>Overview</title> <para>Teiid provides several capabilities for caching data including: materialized views, result set caching, and code table caching. -  These can be used to significantly improve performance in many - situations. </para> - - <section id="cache_hint"> - <title>Cache Hint</title> - <para>Non-update user commands may be preceded with a cache hint, e.g. /*+ cache */ select ..., to inform the engine that the results of command should be cached. The scope of the result, either session or global, will be deteremined automatically from the determinism level of the source queries and functions executed.</para> - <para>ResultSet caching must be enabled for this hint to have an effect.</para> - </section> +  These techniques can be used to significantly improve performance in many + situations.</para> + <para>With the exception of external materialized views, the cached data is accessed through the BufferManager. + For better performance, the BufferManager setting should be adjusted to the memory constraints of your installation.</para> </chapter> Modified: trunk/documentation/caching-guide/src/main/docbook/en-US/content/resultset.xml =================================================================== --- trunk/documentation/caching-guide/src/main/docbook/en-US/content/resultset.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/documentation/caching-guide/src/main/docbook/en-US/content/resultset.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -11,7 +11,7 @@ can yield significant performance gains if users of the system submit the same queries often.</para> <para>Result set caching will cache result sets based on an exact match - of the incoming SQL string. Caching only applies to SELECT, set query, and stored procedure execution + of the incoming SQL string and PreparedStatement parameter values if present. Caching only applies to SELECT, set query, and stored procedure execution statements; it does not apply to SELECT INTO statements, or INSERT, UPDATE, or DELETE statements.</para> <section> Modified: trunk/documentation/caching-guide/src/main/docbook/en-US/main.xml =================================================================== --- trunk/documentation/caching-guide/src/main/docbook/en-US/main.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/documentation/caching-guide/src/main/docbook/en-US/main.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -47,7 +47,8 @@ </bookinfo> <toc /> <xi:include href="content/overview.xml" xmlns:xi="http://www.w3.org/2001/XInclude" /> - <xi:include href="content/matviews.xml" xmlns:xi="http://www.w3.org/2001/XInclude" /> <xi:include href="content/resultset.xml" xmlns:xi="http://www.w3.org/2001/XInclude" /> <xi:include href="content/codetable.xml" xmlns:xi="http://www.w3.org/2001/XInclude" /> + <xi:include href="content/matviews.xml" xmlns:xi="http://www.w3.org/2001/XInclude" /> + <xi:include href="content/cachehint.xml" xmlns:xi="http://www.w3.org/2001/XInclude" /> </book> \ No newline at end of file Modified: trunk/documentation/reference/src/main/docbook/en-US/content/system_schema.xml =================================================================== --- trunk/documentation/reference/src/main/docbook/en-US/content/system_schema.xml 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/documentation/reference/src/main/docbook/en-US/content/system_schema.xml 2010-08-17 13:42:24 UTC (rev 2467) @@ -393,6 +393,128 @@ </informaltable> </section> <section> + <title>SYS.MatViews</title> + <para>This table supplies information about all the materailized views in the virtual database.</para> + <informaltable frame="all"> + <tgroup cols="3"> + <thead> + <row> + <entry> + <para>Column Name</para> + </entry> + <entry> + <para>Type</para> + </entry> + <entry> + <para>Description</para> + </entry> + </row> + </thead> + <tbody> + <row> + <entry> + <para>VDBName</para> + </entry> + <entry> + <para>string</para> + </entry> + <entry> + <para>VDB name</para> + </entry> + </row> + <row> + <entry> + <para>SchemaName</para> + </entry> + <entry> + <para>string</para> + </entry> + <entry> + <para>Schema Name</para> + </entry> + </row> + <row> + <entry> + <para>Name</para> + </entry> + <entry> + <para>string</para> + </entry> + <entry> + <para>Short group name</para> + </entry> + </row> + <row> + <entry> + <para>TargetSchemaName</para> + </entry> + <entry> + <para>string</para> + </entry> + <entry> + <para>Name of the materialized table schema</para> + </entry> + </row> + <row> + <entry> + <para>TargetName</para> + </entry> + <entry> + <para>string</para> + </entry> + <entry> + <para>Name of the materialized table</para> + </entry> + </row> + <row> + <entry> + <para>Valid</para> + </entry> + <entry> + <para>boolean</para> + </entry> + <entry> + <para>True if materialized table is currently valid.</para> + </entry> + </row> + <row> + <entry> + <para>LoadState</para> + </entry> + <entry> + <para>boolean</para> + </entry> + <entry> + <para>The load state, can be one of NEEDS_LOADING, LOADING, LOADED, FAILED_LOAD</para> + </entry> + </row> + <row> + <entry> + <para>Updated</para> + </entry> + <entry> + <para>timestamp</para> + </entry> + <entry> + <para>The timestamp of the last full refresh.</para> + </entry> + </row> + <row> + <entry> + <para>Cardinality</para> + </entry> + <entry> + <para>integer</para> + </entry> + <entry> + <para>The number of rows in the materialized view table.</para> + </entry> + </row> + </tbody> + </tgroup> + </informaltable> + </section> + <section> <title>SYS.Columns</title> <para>This table supplies information about all the elements (columns, tags, attributes, etc) in the virtual database.</para> @@ -1646,6 +1768,28 @@ <para>A single column containing the schemas as clobs.</para> </entry> </row> + <row> + <entry> + <para>refreshMatView</para> + </entry> + <entry> + <para>(string ViewName, boolean Invalidate)</para> + </entry> + <entry> + <para>An return return value, RowsUpdated. See the Caching Guide for more.</para> + </entry> + </row> + <row> + <entry> + <para>refreshMatViewRow</para> + </entry> + <entry> + <para>(string ViewName, object Key)</para> + </entry> + <entry> + <para>An return return value, RowsUpdated. See the Caching Guide for more.</para> + </entry> + </row> </tbody> </tgroup> </informaltable> Modified: trunk/engine/src/main/java/org/teiid/common/buffer/STree.java =================================================================== --- trunk/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/engine/src/main/java/org/teiid/common/buffer/STree.java 2010-08-17 13:42:24 UTC (rev 2467) @@ -212,7 +212,7 @@ } else { SearchResult result = places.removeLast(); Object value = (i == 0 ? tuple : page); - page = insert(key, result, places.peekLast(), value); + page = insert(key, result, places.peekLast(), value, mode == InsertMode.ORDERED); } } return null; @@ -222,19 +222,13 @@ return tuple.subList(0, keyLength); } - SPage insert(List k, SearchResult result, SearchResult parent, Object value) throws TeiidComponentException { + SPage insert(List k, SearchResult result, SearchResult parent, Object value, boolean ordered) throws TeiidComponentException { SPage page = result.page; int index = -result.index - 1; if (result.values.getTuples().size() == pageSize) { boolean leaf = !(value instanceof SPage); SPage nextPage = new SPage(this, leaf); TupleBatch nextValues = nextPage.getValues(); - nextValues.getTuples().addAll(result.values.getTuples().subList(pageSize/2, pageSize)); - result.values.getTuples().subList(pageSize/2, pageSize).clear(); - if (!leaf) { - nextPage.children.addAll(page.children.subList(pageSize/2, pageSize)); - page.children.subList(pageSize/2, pageSize).clear(); - } nextPage.next = page.next; nextPage.prev = page; if (nextPage.next != null) { @@ -242,18 +236,30 @@ } page.next = nextPage; boolean inNext = false; - if (index <= pageSize/2) { - setValue(index, k, value, result.values, page); + if (!ordered) { + //split the values + nextValues.getTuples().addAll(result.values.getTuples().subList(pageSize/2, pageSize)); + result.values.getTuples().subList(pageSize/2, pageSize).clear(); + if (!leaf) { + nextPage.children.addAll(page.children.subList(pageSize/2, pageSize)); + page.children.subList(pageSize/2, pageSize).clear(); + } + if (index <= pageSize/2) { + setValue(index, k, value, result.values, page); + } else { + inNext = true; + setValue(index - pageSize/2, k, value, nextValues, nextPage); + } + page.setValues(result.values); + if (parent != null) { + List min = nextPage.getValues().getTuples().get(0); + SPage.correctParents(parent.page, min, page, nextPage); + } } else { inNext = true; - setValue(index - pageSize/2, k, value, nextValues, nextPage); + setValue(0, k, value, nextValues, nextPage); } nextPage.setValues(nextValues); - page.setValues(result.values); - if (parent != null) { - List min = nextPage.getValues().getTuples().get(0); - SPage.correctParents(parent.page, min, page, nextPage); - } if (inNext) { page = nextPage; } Modified: trunk/engine/src/test/java/org/teiid/cache/TestDefaultCache.java =================================================================== --- trunk/engine/src/test/java/org/teiid/cache/TestDefaultCache.java 2010-08-16 20:06:21 UTC (rev 2466) +++ trunk/engine/src/test/java/org/teiid/cache/TestDefaultCache.java 2010-08-17 13:42:24 UTC (rev 2467) @@ -26,18 +26,19 @@ import org.junit.Test; +@SuppressWarnings("nls") public class TestDefaultCache { @Test public void testExpiration() throws InterruptedException { - DefaultCache<Integer, Integer> cache = new DefaultCache<Integer, Integer>("foo", 2, 50); + DefaultCache<Integer, Integer> cache = new DefaultCache<Integer, Integer>("foo", 2, 70); cache.put(1, 1); - Thread.sleep(60); + Thread.sleep(100); assertNull(cache.get(1)); cache.put(2, 2); - Thread.sleep(30); + Thread.sleep(50); cache.put(3, 3); assertNotNull(cache.get(2)); - Thread.sleep(40); + Thread.sleep(50); cache.put(4, 4); //preferred to purge 2 instead of 3 assertNotNull(cache.get(3));