[JBoss JIRA] (TEIID-2411) Add schema name to metadata validation messages
by Paul Lysak (JIRA)
Paul Lysak created TEIID-2411:
---------------------------------
Summary: Add schema name to metadata validation messages
Key: TEIID-2411
URL: https://issues.jboss.org/browse/TEIID-2411
Project: Teiid
Issue Type: Enhancement
Components: Query Engine
Affects Versions: 8.2
Reporter: Paul Lysak
Assignee: Steven Hawkins
Currently metadata validation messages don't contain schema name, so when working with lot of schemas it may be difficult to find out problematic element. For example, if table SOME_TABLE has no columns then
org.teiid.query.metadata.MetadataValidator.MinimalMetadata.execute() will log following error message:
TEIID31071 Invalid table; Table SOME_TABLE has no columns defined
And there's no ability to detect later to which schema problematic table belongs.
I suggest to append schema name - either by simple concatenation in MetadataValidator.java:
metadataValidator.log(report, model, QueryPlugin.Util.gs(QueryPlugin.Event.TEIID31071, scmena.getName()+"."+t.getName()));
or by changing message bundle i18n.properties:
TEIID31071=Invalid table; Table {0}.{1} has no columns defined
and providing message parameters in MetadataValidator.java:
metadataValidator.log(report, model, QueryPlugin.Util.gs(QueryPlugin.Event.TEIID31071, scmena.getName(), t.getName()));
There are also another validation messages in MetadataValidator.java which could be improved in similar way.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2410) issues with output buffer blocking
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2410?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-2410.
-----------------------------------
Resolution: Done
updated the logic to be safe for local/embedded connections (although there is room for improvement when not using the calling thread, but that is a corner case), prevented blocking on scrollable results, and prevented process work from performing large evictions.
> issues with output buffer blocking
> ----------------------------------
>
> Key: TEIID-2410
> URL: https://issues.jboss.org/browse/TEIID-2410
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 7.7
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> when a local connection (regardless of using the calling thread) blocks on the output buffer (which cannot happen with forward only and using the calling thread, but due to another issue can occur with a scroll insensitive result) no corresponding process thread is awakened to process pending work.
> Output blocking should also be disabled for non-forward only results. This was disabled at some point by a refactoring to reuse the output buffer from the plan.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2410) issues with output buffer blocking
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2410?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-2410:
----------------------------------
Summary: issues with output buffer blocking (was: add tuning for lots of active plans)
Issue Type: Bug (was: Quality Risk)
Fix Version/s: 8.3
Affects Version/s: 7.7
Description:
when a local connection (regardless of using the calling thread) blocks on the output buffer (which cannot happen with forward only and using the calling thread, but due to another issue can occur with a scroll insensitive result) no corresponding process thread is awakened to process pending work.
Output blocking should also be disabled for non-forward only results. This was disabled at some point by a refactoring to reuse the output buffer from the plan.
was:When an active plan is not processing it will still be holding memory against the buffer manager reserve count.
> issues with output buffer blocking
> ----------------------------------
>
> Key: TEIID-2410
> URL: https://issues.jboss.org/browse/TEIID-2410
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 7.7
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> when a local connection (regardless of using the calling thread) blocks on the output buffer (which cannot happen with forward only and using the calling thread, but due to another issue can occur with a scroll insensitive result) no corresponding process thread is awakened to process pending work.
> Output blocking should also be disabled for non-forward only results. This was disabled at some point by a refactoring to reuse the output buffer from the plan.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2409) To add execution id into data src command_log
by Jack Ma (JIRA)
[ https://issues.jboss.org/browse/TEIID-2409?page=com.atlassian.jira.plugin... ]
Jack Ma commented on TEIID-2409:
--------------------------------
Ramesh, thanks for the explanation. If teiid could just adds a log at command_log, which could let us determining how long the user query stays at waiting queue, that will be very helpfully for us to do the performance tuning up.
> To add execution id into data src command_log
> ---------------------------------------------
>
> Key: TEIID-2409
> URL: https://issues.jboss.org/browse/TEIID-2409
> Project: Teiid
> Issue Type: Feature Request
> Affects Versions: 8.2
> Reporter: Jack Ma
> Assignee: Steven Hawkins
>
> When we try to analyze the command_log, for some user query, it push multiple data source queries to same data source, in this case, both request id and source command id are same, so it is impossible to pair those data source queries from the command_log (please see discussion: https://community.jboss.org/message/799183#799183). Adding the execution id into the data source command_log should make following combination key unique (request id + source command id + execution id)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2408:
-------------------------------------
OK, I agree. I was even contemplating to show that stat for last 30 min too. Tools like RHQ can probably do a better job of these over time period.
> statistics of teiid query queuing information
> ---------------------------------------------
>
> Key: TEIID-2408
> URL: https://issues.jboss.org/browse/TEIID-2408
> Project: Teiid
> Issue Type: Feature Request
> Components: AdminApi
> Affects Versions: 8.2
> Reporter: Jack Ma
> Assignee: Ramesh Reddy
> Fix For: 8.3
>
>
> At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up,
> Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2408:
---------------------------------------
I'm going to remove the avg-waitplan stat. A server-side average like that will quickly become meaningless. A threshold may be an interesting value to check against, but otherwise we're better off allowing that information to be derived from the commandlog/events - we do have a plan event currently at the debug level.
> statistics of teiid query queuing information
> ---------------------------------------------
>
> Key: TEIID-2408
> URL: https://issues.jboss.org/browse/TEIID-2408
> Project: Teiid
> Issue Type: Feature Request
> Components: AdminApi
> Affects Versions: 8.2
> Reporter: Jack Ma
> Assignee: Ramesh Reddy
> Fix For: 8.3
>
>
> At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up,
> Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ]
Ramesh Reddy resolved TEIID-2408.
---------------------------------
Resolution: Done
Added
# active-plans-count => Current active plans in query engine
# waiting-plans-count => Waiting plans to process in queue
# max-waitplan-watermark => Max amount time any plan waited in queue
# avg-waitplan-time-milli => avg amount time the plans that were added wait queue, has spent in the queue waiting
{code}
[standalone@localhost:9999 /] /subsystem=teiid:engine-statistics
{
"outcome" => "success",
"result" => {
"session-count" => 25,
"total-memory-inuse-kb" => 0L,
"total-memory-inuse-active-plans-kb" => 1470L,
"buffermgr-disk-write-count" => 0L,
"buffermgr-disk-read-count" => 0L,
"buffermgr-cache-write-count" => 0L,
"buffermgr-cache-read-count" => 0L,
"buffermgr-diskspace-used-mb" => 0L,
"active-plans-count" => 2,
"waiting-plans-count" => 0,
"max-waitplan-watermark" => 0,
"avg-waitplan-time-milli" => 0L
}
}
{code}
> statistics of teiid query queuing information
> ---------------------------------------------
>
> Key: TEIID-2408
> URL: https://issues.jboss.org/browse/TEIID-2408
> Project: Teiid
> Issue Type: Feature Request
> Components: AdminApi
> Affects Versions: 8.2
> Reporter: Jack Ma
> Assignee: Ramesh Reddy
> Fix For: 8.3
>
>
> At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up,
> Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2404:
-------------------------------------
{code}
[standalone@localhost:9999 /] /subsystem=teiid:engine-statistics
{
"outcome" => "success",
"result" => {
"session-count" => 25,
"total-memory-inuse-kb" => 0L,
"total-memory-inuse-active-plans-kb" => 1470L,
"buffermgr-disk-write-count" => 0L,
"buffermgr-disk-read-count" => 0L,
"buffermgr-cache-write-count" => 0L,
"buffermgr-cache-read-count" => 0L,
"buffermgr-diskspace-used-mb" => 0L,
"active-plans-count" => 2,
"waiting-plans-count" => 0,
"max-waitplan-watermark" => 0,
"avg-waitplan-time-milli" => 0L
}
}
{code}
> Expose buffermanager monitoring properties
> ------------------------------------------
>
> Key: TEIID-2404
> URL: https://issues.jboss.org/browse/TEIID-2404
> Project: Teiid
> Issue Type: Feature Request
> Components: Server
> Affects Versions: 8.3
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 8.3
>
>
> We should expose:
> BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage.
> A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size.
> Also the BufferManagerImpl could report the total estimate of objects held in memory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ]
Ramesh Reddy resolved TEIID-2404.
---------------------------------
Resolution: Done
Couple other stats that are also part of this are the number of reads and writes into soft cache based store. The disk store is always fronted by soft referenced memory store, so, write to disk not necessarily correspond with read, as the content may be available in the soft-reference store.
> Expose buffermanager monitoring properties
> ------------------------------------------
>
> Key: TEIID-2404
> URL: https://issues.jboss.org/browse/TEIID-2404
> Project: Teiid
> Issue Type: Feature Request
> Components: Server
> Affects Versions: 8.3
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 8.3
>
>
> We should expose:
> BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage.
> A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size.
> Also the BufferManagerImpl could report the total estimate of objects held in memory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (TEIID-2409) To add execution id into data src command_log
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2409?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2409:
-------------------------------------
IMO, there is not much use in adding multiple of these calls, in TEIID-2408 we are adding the couple wait queue stats, that should tell you if there is waiting of plans occurring, and their average times. The general goal is to make wait times less or not existent. So, you need configure for such scenario.
When PLAN USER COMMAND is printed, the planning is already done. So, from start to plan print outs is Teiid overhead. Also, did you look at getPlanDescription() on your statement object, it has some statistics on them, may be we could any fine grained stats there instead.
> To add execution id into data src command_log
> ---------------------------------------------
>
> Key: TEIID-2409
> URL: https://issues.jboss.org/browse/TEIID-2409
> Project: Teiid
> Issue Type: Feature Request
> Affects Versions: 8.2
> Reporter: Jack Ma
> Assignee: Steven Hawkins
>
> When we try to analyze the command_log, for some user query, it push multiple data source queries to same data source, in this case, both request id and source command id are same, so it is impossible to pair those data source queries from the command_log (please see discussion: https://community.jboss.org/message/799183#799183). Adding the execution id into the data source command_log should make following combination key unique (request id + source command id + execution id)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months