February 2013 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-2368) Dynamic VDB deployment fails when using importer.importIndexes in the case of a index containing a hidden column

by SBS JIRA Integration (JIRA)

[ https://issues.jboss.org/browse/TEIID-2368?page=com.atlassian.jira.plugin... ] SBS JIRA Integration updated TEIID-2368: ---------------------------------------- Forum Reference: https://community.jboss.org/message/795305#795305, https://community.jboss.org/message/795313#795313 (was: https://community.jboss.org/message/795305#795305) > Dynamic VDB deployment fails when using importer.importIndexes in the case of a index containing a hidden column > ---------------------------------------------------------------------------------------------------------------- > > Key: TEIID-2368 > URL: https://issues.jboss.org/browse/TEIID-2368 > Project: Teiid > Issue Type: Feature Request > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Steven Hawkins > Fix For: 8.3 > > > We are trying to use dynamic VDB. For better performance, we use <property name="importer.importIndexes" value="true"/> to input the index for an oracle datasource. > > But vdb deployment failed with following error: > [TEIID60011 No column found with name SYS_NC00032$] > > Debugging through teiid code, it looks like that if the index contains the hidden column (like SYS_NC00032), then index loading failed, teiid was trying to call getColumnByName() for those hidden column which does not exist in the column map. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2409) To add execution id into data src command_log

by Jack Ma (JIRA)

[ https://issues.jboss.org/browse/TEIID-2409?page=com.atlassian.jira.plugin... ] Jack Ma commented on TEIID-2409: -------------------------------- Also, could we add some command log to show when the USER QUERY comes out from the waiting queue and also when PLAN USER COMMAND is started? So we could know what is total waiting time at teiid waiting queue and what is the time for the planning. > To add execution id into data src command_log > --------------------------------------------- > > Key: TEIID-2409 > URL: https://issues.jboss.org/browse/TEIID-2409 > Project: Teiid > Issue Type: Feature Request > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Steven Hawkins > > When we try to analyze the command_log, for some user query, it push multiple data source queries to same data source, in this case, both request id and source command id are same, so it is impossible to pair those data source queries from the command_log (please see discussion: https://community.jboss.org/message/799183#799183). Adding the execution id into the data source command_log should make following combination key unique (request id + source command id + execution id) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2410) add tuning for lots of active plans

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2410?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-2410: ---------------------------------- Description: When an active plan is not processing it will still be holding memory against the buffer manager reserve count. (was: When an active plan is not processing it will still be holding memory against the buffer manager reserve count. Also even non-active, but not closed plans can still hold their reservation even though a batch may not be held by the workitem.) I made an initial misdiagnosis here. non-active plans are not holding their reserves - even if they are holding a batch for the forward only logic (which seems like we need to account for). The real problem was that with useCallingThread=false and plans that cause output buffer blocking we will not have the proper number of worker threads active, which will slow processing to a crawl as new plans back-up waiting for older plans to finish processing. > add tuning for lots of active plans > ----------------------------------- > > Key: TEIID-2410 > URL: https://issues.jboss.org/browse/TEIID-2410 > Project: Teiid > Issue Type: Quality Risk > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > > When an active plan is not processing it will still be holding memory against the buffer manager reserve count. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2410) add tuning for lots of active plans

by Steven Hawkins (JIRA)

Steven Hawkins created TEIID-2410: ------------------------------------- Summary: add tuning for lots of active plans Key: TEIID-2410 URL: https://issues.jboss.org/browse/TEIID-2410 Project: Teiid Issue Type: Quality Risk Components: Query Engine Reporter: Steven Hawkins Assignee: Steven Hawkins When an active plan is not processing it will still be holding memory against the buffer manager reserve count. Also even non-active, but not closed plans can still hold their reservation even though a batch may not be held by the workitem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2404: --------------------------------------- Here are the refined properties getMemoryBufferUsedKB - derived from the blocks used, which is completely accurate. getHeapMemoryInUseByActivePlansKB - refined and renamed getMemoryInUseByActivePlansKB, however this is an intentional over estimate and can easily report a value larger than the vm size. We are using it as a measure of maximum memory pressure to ensure that we don't trigger an OOM by holding references in the buffermanager. getHeapCacheMemoryInUseKB - refined and renamed getTotalMemoryInUseKB, however here again this is likely an overestimate - but it won't be as bad as getMemoryInUseByActivePlansKB since the batches will be sized based upon the objects they hold. > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2404: ------------------------------------- Changed to below, but does not look like runtime.totalMemory is reliable. {code:java} private long calculateMaxReserveKB() { if (getMaxReservedKb() < 0) { long memoryInGB = Runtime.getRuntime().totalMemory()/(1024*1024); return new Double(((1024-300) * 0.5) + ((memoryInGB-1) * 1024 * 0.75)).longValue(); } return getMaxReservedKb(); } public long getTotalMemoryInUseKB() { return bufferMgr.getActiveBatchBytes()/1024 + calculateMaxReserveKB() - bufferMgr.getMaxReserveKB(); } public long getMemoryInUseByActivePlansKB() { return ((calculateMaxReserveKB()*1024) - bufferMgr.getReserveBatchBytes())/1024; } {code} > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2409) To add execution id into data src command_log

by SBS JIRA Integration (JIRA)

[ https://issues.jboss.org/browse/TEIID-2409?page=com.atlassian.jira.plugin... ] SBS JIRA Integration updated TEIID-2409: ---------------------------------------- Forum Reference: https://community.jboss.org/message/799128#799128 > To add execution id into data src command_log > --------------------------------------------- > > Key: TEIID-2409 > URL: https://issues.jboss.org/browse/TEIID-2409 > Project: Teiid > Issue Type: Feature Request > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Steven Hawkins > > When we try to analyze the command_log, for some user query, it push multiple data source queries to same data source, in this case, both request id and source command id are same, so it is impossible to pair those data source queries from the command_log (please see discussion: https://community.jboss.org/message/799183#799183). Adding the execution id into the data source command_log should make following combination key unique (request id + source command id + execution id) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2409) To add execution id into data src command_log

by Jack Ma (JIRA)

Jack Ma created TEIID-2409: ------------------------------ Summary: To add execution id into data src command_log Key: TEIID-2409 URL: https://issues.jboss.org/browse/TEIID-2409 Project: Teiid Issue Type: Feature Request Affects Versions: 8.2 Reporter: Jack Ma Assignee: Steven Hawkins When we try to analyze the command_log, for some user query, it push multiple data source queries to same data source, in this case, both request id and source command id are same, so it is impossible to pair those data source queries from the command_log (please see discussion: https://community.jboss.org/message/799183#799183). Adding the execution id into the data source command_log should make following combination key unique (request id + source command id + execution id) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2404: ------------------------------------- SteveH, I put in formulas as above and commited basic structure to expose these properties. However, I do not think they currently correctly represent the correct value based on default values. I was not sure how to adjust for that. Can you redo/fix those methods in BufferServiceImpl class? Then you can resolve this JIRA. Thx > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 3 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ] Ramesh Reddy updated TEIID-2408: -------------------------------- Assignee: Ramesh Reddy (was: Steven Hawkins) Fix Version/s: 8.3 Complexity: Low > statistics of teiid query queuing information > --------------------------------------------- > > Key: TEIID-2408 > URL: https://issues.jboss.org/browse/TEIID-2408 > Project: Teiid > Issue Type: Feature Request > Components: AdminApi > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Ramesh Reddy > Fix For: 8.3 > > > At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up, > Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 3 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues February 2013