From teiid-commits at lists.jboss.org Tue Sep 7 23:27:09 2010
Content-Type: multipart/mixed; boundary="===============6509674920851319814=="
MIME-Version: 1.0
From: teiid-commits at lists.jboss.org
To: teiid-commits at lists.jboss.org
Subject: [teiid-commits] teiid SVN: r2541 - in branches/7.1.x/documentation:
admin-guide/src/main/docbook/en-US/content and 1 other directories.
Date: Tue, 07 Sep 2010 23:27:09 -0400
Message-ID: <201009080327.o883R9Be007220@svn01.web.mwc.hst.phx2.redhat.com>
--===============6509674920851319814==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Author: shawkins
Date: 2010-09-07 23:27:08 -0400 (Tue, 07 Sep 2010)
New Revision: 2541
Modified:
branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/admin_gu=
ide.xml
branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content/=
performance.xml
branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/sc=
alar_functions.xml
Log:
TEIID-1215 adding a performance chapter to the admin guide
Modified: branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/a=
dmin_guide.xml
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/admin_g=
uide.xml 2010-09-08 00:37:02 UTC (rev 2540)
+++ branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/admin_g=
uide.xml 2010-09-08 03:27:08 UTC (rev 2541)
@@ -50,6 +50,7 @@
+ =
Modified: branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/c=
ontent/performance.xml
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content=
/performance.xml 2010-09-08 00:37:02 UTC (rev 2540)
+++ branches/7.1.x/documentation/admin-guide/src/main/docbook/en-US/content=
/performance.xml 2010-09-08 03:27:08 UTC (rev 2541)
@@ -1,28 +1,127 @@
-
\ No newline at end of file
+
+
+%CustomDTD;
+]>
+
+ Performance Tuning
+
+ Memory Management
+ The BufferManager is responsible for tracking both memory and disk=
usage
+ by Teiid. Configuring the BufferManager properly is one of the most
+ important parts of ensuring high performance. See the &jboss-beans;
+ file for all BufferManager settings.
+
+
+ The Teiid engine uses batching to reduce the number of memory
+ rows processed at a given time. The batch sizes may be adjusted to larg=
er values if few clients will be
+ accessing the Teiid server simultaneously. =
+
+
+ The maxReserveBatchColumns
+ setting determines the total number of batches (with a max of
+ processorBatchSize rows) multiplied by their column width
+ that can be held in memory directly by the BufferManager.
+ This number does not include persistent batches held by soft (such as
+ index pages) or weak references. When your installation can dedicate
+ more memory to Teiid, consider increasing this value in proportion to t=
he number of gigabytes you
+ wish Teiid to use - e.g. 2GB on a 32 bit VM would double the value to
+ 32768. For 64 bit VMs you should use a value of approximately 11000
+ per GB.
+ The BufferManager automatically triggers the use of a canonical
+ value cache when more than 25% of the reserve is in use.
+ This can dramatically cut the memory usage in situations where similar
+ value sets are being read through Teiid, but does introduce a lookup co=
st.
+ If you are processing large (100s of MBs) of highly unique datasets
+ through Teiid, you should consider =
disabling value caching since it
+ will not significantly reduce memory consumption.
+
+
+ Each intermediate result buffer, temporary LOB, and temporary table
+ is stored in its own set of buffer files,
+ where an individual file is
+ limited to maxFileSize megabytes.
+ Consider increasing the storage space available to all such
+ files maxBufferSpace
+ if your installation makes use of internal materialization, makes
+ heavy use of SQL/XML, or processes large row counts.
+
+
+
+ Threading
+ Socket threads are configured for each transport. =
+ They handle NIO non-blocking IO operations as well as directly servicing=
any operation that can run without blocking.
+ For longer running operations, the socket threads queue with work =
the query engine. =
+ The query engine has two settings that determine its thread utilization.=
=
+ maxThreads sets the total number of threads available for q=
uery engine work (processing plans, transaction control operations, process=
ing source queries, etc.). =
+ You should consider increasing the maximum threads on systems with a lar=
ge number of available processors and/or when it's common to issue non-tran=
sactional queries with that =
+ issue a large number of concurrent source requests.
+ maxActivePlans, which should always be smaller than maxThre=
ads, sets the number of the maxThreads =
+ that should be used for user query processing. Increasing the maxActive=
Plans should be considered for workloads with a high number of long =
+ running queries and/or systems with a large number of available processo=
rs. If memory issues arise from increasing the max threads and the
+ max active plans, then consider decreasing the processor/connector batch=
sizes to limit the base number of memory rows consumed by each plan.
+
+
+ Cache Tuning
+ Caching can be tuned for cached result (including user query resul=
ts and procedure results) and prepared plans (including user and stored pro=
cedure plans). =
+ Even though it is possible to disable or otherwise severely constrain th=
ese caches, this would probably never be done in practice as it would lead =
to poor performance.
+ Cache statistics can be obtained through the Admin Console =
or Adminshell. The statistics can be used to help tune cache parameters an=
d ensure a hit ratio.
+ Plans are currently fully held in memory and may have a significan=
t memory footprint. When making extensive use of prepared statements and/o=
r virtual procedures, =
+ the size of the plan cache may be increased proportionally to number of =
GB intended for use by Teiid.
+ While the result cache parameters control the cache result =
entries (max number, eviction, etc.), the result batches themselves are acc=
essed through the BufferManager.
+ If the size of the result cache is increased, you may need to tune the B=
ufferManager configuration to ensure there is enough buffer space.
+
+
+
+ Socket Transports
+ Teiid separates the configuration of its socket transports for
+ JDBC, ODBC, and Admin access.
+ Typical installations will not need to
+ adjust the default thread and buffer
+ size settings. At this time, ODBC
+ queries are executed synchronously from the socket thread.
+ Simultaneous long-running queries may exhaust the available threads.
+ Consider increasing the default max threads (15) for ODBC if you
+ expect a higher concurrent load of long-running queries.
+
+
+ LOBs
+ LOBs and XML documents are streamed from the Teiid Server to the
+ Teiid JDBC API.=C2=A0=C2=A0
+ Normally, these values are not materialized in the
+ server memory - avoiding potential out-of-memory issues.
+ When using style sheets, or XQuery, whole XML documents must be
+ materialized on the server.
+ Even when using the XMLQuery or XMLTable functions and
+ document projection is applied, memory issues may occur for large
+ documents.
+
+
+ LOBs are broken into pieces when being created and streamed.=C2=A0
+ The maximum size of each piece when fetched by the client can be
+ configured with the "lobChunkSizeInKB"
+ property in the &jboss-beans; file. The default value is 100 KB.
+ When dealing with extremely large LOBs, you may consider increasing this
+ value to decrease the amount of round-trips to stream the result.
+ Setting the value too high may cause the server or client to have
+ memory issues.
+
+ Source LOB values are typically accessed by reference, rather
+ than having the value copied to a temporary location.
+ Thus care must be taken to ensure that source LOBs are returned in a
+ memory-safe manner. =
+
+
+
+ Other Considerations
+ When
+ using Teiid in a development environment, you may consider setting
+ the maxSourceRows property in the &jboss-beans;
+ file to reasonably small level value (e.g. 10000) to prevent large
+ amounts of data from being pulled from sources.
+ Leaving the exceptionOnMaxSourceRows set to true will alert the develop=
er
+ through an exception that an attempt was made to retrieve more than
+ the specified number of rows.
+
+
+
\ No newline at end of file
Modified: branches/7.1.x/documentation/reference/src/main/docbook/en-US/con=
tent/scalar_functions.xml
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/s=
calar_functions.xml 2010-09-08 00:37:02 UTC (rev 2540)
+++ branches/7.1.x/documentation/reference/src/main/docbook/en-US/content/s=
calar_functions.xml 2010-09-08 03:27:08 UTC (rev 2541)
@@ -1968,7 +1968,8 @@
xquery in string. Return value is xml.XMLQUERY is part of the SQL/XML 2006 specification.
See also XMLTABLE
- A technique known as document projection is used =
to reduce the memory footprint of the context item document. Only the part=
s of the document needed by the xquery will be loaded into memory.
+ A technique known as document projection is used =
to reduce the memory footprint of the context item document. =
+ Only the parts of the document needed by the XQuery path expr=
essions will be loaded into memory. Since document projection analysis use=
s all relevant path expressions, even 1 expression that could potentially u=
se many nodes, e.g. //x rather than /a/b/x will cause a larger memory footp=
rint.
=
--===============6509674920851319814==--