February 2013 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information

by Van Halbert (JIRA)

[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ] Van Halbert commented on TEIID-2408: ------------------------------------ Pulling using monitoring to check the levels would be good. This would work perfect in JON to provide history of the levels over time. And to watch for peak period. > statistics of teiid query queuing information > --------------------------------------------- > > Key: TEIID-2408 > URL: https://issues.jboss.org/browse/TEIID-2408 > Project: Teiid > Issue Type: Feature Request > Components: AdminApi > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Steven Hawkins > > At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up, > Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2404: ------------------------------------- so, Total Amount of memory in use = activBatchBytes + (initial maxReserveBytes) - maxReserveBytes Amount of memory in use by active plans = (initial maxReserveBytes) - reserveBatchBytes What would be "blocksInUse.getBitsSet() * BLOCK_SIZE" represent? Is buffer memory used by FileStore? > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2404: --------------------------------------- 4) maxReserveBytes is a working measure of how much space is left over out of the initial maxReserveKB setting after accounting for soft memory references (which aren't counted toward the activeBatchBytes) and batch overhead. So the "Amount of Memory" we believe to be in use by the buffermanager object hard/soft references (not including the memory buffer / disk) is: activBatchBytes + (initial maxReserveBytes) - maxReserveBytes However we would need to add a reference to the initial maxReserveBytes setting to compute this. reserveBatchBytes is the amount of available space (out of the initial maxReserveKB) to be reserved by running plans. This value can drop below zero as plans can force memory reservations beyond the max. So the "amount of memory that plans have requested" is: (initial maxReserveBytes) - reserveBatchBytes The object/batches represented by reserveBatchBytes may (potentially as a weak or soft reference) or may not be held by the buffermanager. > I do not think I understand what it means by "BufferFrontedCacheStore concurrent bits set X block size." There are ConcurrentBitSet's that track the blocks of memory in use. There's one for blocks and one for inodes (which is mostly negligible). So the amount of the memory buffer currently in use would be: blocksInUse.getBitsSet() * BLOCK_SIZE where the memory block size is currently fixed at 8KB. > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2404: ------------------------------------- Amount of Memory in Use = activBatchBytes + maxReserveBytes > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2408: ------------------------------------- @Van - you mean as push from Teiid? If yes that is not possible. You can always make request and pull the stats when the feature is implemented > statistics of teiid query queuing information > --------------------------------------------- > > Key: TEIID-2408 > URL: https://issues.jboss.org/browse/TEIID-2408 > Project: Teiid > Issue Type: Feature Request > Components: AdminApi > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Steven Hawkins > > At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up, > Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information

by Van Halbert (JIRA)

[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ] Van Halbert commented on TEIID-2408: ------------------------------------ Can there be a monitoring option to see the current levels? > statistics of teiid query queuing information > --------------------------------------------- > > Key: TEIID-2408 > URL: https://issues.jboss.org/browse/TEIID-2408 > Project: Teiid > Issue Type: Feature Request > Components: AdminApi > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Steven Hawkins > > At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up, > Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2404: ------------------------------------- # reads from disk = BufferFrontedCacheStore.getStorageReads # writes to disk = BufferFrontedCacheStore.getStorageWrites # disk space used = BufferServiceImpl.getUserBufferSpace # Amount in soft cache, that can be reclaimed in gc = maxReserveBytes I do not think I understand what it means by "BufferFrontedCacheStore concurrent bits set X block size." I am not sure if knowing the # objects in memory is any good use. > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Ramesh Reddy edited comment on TEIID-2404 at 2/22/13 11:37 AM: --------------------------------------------------------------- # reads from disk = BufferFrontedCacheStore.getStorageReads # writes to disk = BufferFrontedCacheStore.getStorageWrites # disk space used = BufferServiceImpl.getUserBufferSpace # Amount in soft cache, that can be reclaimed in gc = maxReserveBytes I do not think I understand what it means by "BufferFrontedCacheStore concurrent bits set X block size." I am not sure if knowing the # objects in memory is any good use. was (Author: rareddy): # reads from disk = BufferFrontedCacheStore.getStorageReads # writes to disk = BufferFrontedCacheStore.getStorageWrites # disk space used = BufferServiceImpl.getUserBufferSpace # Amount in soft cache, that can be reclaimed in gc = maxReserveBytes I do not think I understand what it means by "BufferFrontedCacheStore concurrent bits set X block size." I am not sure if knowing the # objects in memory is any good use. > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2404) Expose buffermanager monitoring properties

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2404?page=com.atlassian.jira.plugin... ] Ramesh Reddy edited comment on TEIID-2404 at 2/22/13 11:38 AM: --------------------------------------------------------------- # reads from disk = BufferFrontedCacheStore.getStorageReads # writes to disk = BufferFrontedCacheStore.getStorageWrites # disk space used = BufferServiceImpl.getUserBufferSpace # Amount in soft cache, that can be reclaimed in gc = maxReserveBytes I do not think I understand what it means by "BufferFrontedCacheStore concurrent bits set X block size." I am not sure if knowing the # objects in memory is any good use. was (Author: rareddy): # reads from disk = BufferFrontedCacheStore.getStorageReads # writes to disk = BufferFrontedCacheStore.getStorageWrites # disk space used = BufferServiceImpl.getUserBufferSpace # Amount in soft cache, that can be reclaimed in gc = maxReserveBytes I do not think I understand what it means by "BufferFrontedCacheStore concurrent bits set X block size." I am not sure if knowing the # objects in memory is any good use. > Expose buffermanager monitoring properties > ------------------------------------------ > > Key: TEIID-2404 > URL: https://issues.jboss.org/browse/TEIID-2404 > Project: Teiid > Issue Type: Feature Request > Components: Server > Affects Versions: 8.3 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Fix For: 8.3 > > > We should expose: > BufferServiceImpl.getUserBufferSpace - the number of megs currently used by the buffermanager disk storage. > A rough total of the memory buffer space used can be added, which would be from the BufferFrontedCacheStore concurrent bits set X block size. > Also the BufferManagerImpl could report the total estimate of objects held in memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2408) statistics of teiid query queuing information

by SBS JIRA Integration (JIRA)

[ https://issues.jboss.org/browse/TEIID-2408?page=com.atlassian.jira.plugin... ] SBS JIRA Integration updated TEIID-2408: ---------------------------------------- Forum Reference: https://community.jboss.org/message/798980#798980 > statistics of teiid query queuing information > --------------------------------------------- > > Key: TEIID-2408 > URL: https://issues.jboss.org/browse/TEIID-2408 > Project: Teiid > Issue Type: Feature Request > Components: AdminApi > Affects Versions: 8.2 > Reporter: Jack Ma > Assignee: Steven Hawkins > > At currently version of teiid, there is no teiid queuing information available from either teiid api or log or jmx, like how many query plans are still in the waiting queue, how many are in the active queue etc. .Thinking those queuing information is pretty critical for performance tuning up, > Based on the Ramesh's suggestion, creating this request for probably a statistics max water mark in the waiting queue, and average time spent in waiting queue etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 10 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues February 2013