January 2014 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-2813) Report translator CPU time

by Mark Addleman (JIRA)

[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ] Mark Addleman commented on TEIID-2813: -------------------------------------- In my experience, the overhead of grabbing CPU time is a little higher than getting clock time but the only time this matters is in tight-loop situations. Since the changes apply to fairly heavy-weight methods anyway (particularly, handleBatch() versus next()), I don't think overhead is much of an issue. If you want, I can introduce some switch. > Report translator CPU time > -------------------------- > > Key: TEIID-2813 > URL: https://issues.jboss.org/browse/TEIID-2813 > Project: Teiid > Issue Type: Feature Request > Reporter: Mark Addleman > Assignee: Steven Hawkins > > The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2813) Report translator CPU time

by Mark Addleman (JIRA)

[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ] Mark Addleman commented on TEIID-2813: -------------------------------------- There is a small discussion of this at https://community.jboss.org/thread/236413 I want to know the accumulated CPU from the translator's execute() and next() calls. I agree that nearly all translators are IO bound but some of our translators take on non-trivial CPU work to re-orient data that comes out of the source. In order to better direct our tuning efforts, having CPU time reported against the translator is useful. Ideally, Teiid would also report the total CPU required to process a user query so we can make query tuning decisions and, perhaps, supports* decisions. It is possible that even though a source can handle filtering, Teiid may be able to process the filter more efficiently - we have some very odd data sources. At this time, we don't *need* CPU time at the user-query level so this JIRA only addresses the translator. I was planning on reviewing the code to see how difficult it would be to add instrumentation at the higher level > Report translator CPU time > -------------------------- > > Key: TEIID-2813 > URL: https://issues.jboss.org/browse/TEIID-2813 > Project: Teiid > Issue Type: Feature Request > Reporter: Mark Addleman > Assignee: Steven Hawkins > > The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2790) Add more options for costing during import

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2790?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2790: --------------------------------------- > I thought we have extension properties for the catalog/schema, if not adding them is good idea on the table. No, nor would any hand created metadata / designer vdb contain them. > I say we make available for execution through system procedure execution for ad-hoc Yes, that is where this issue dove tails with TEIID-245 > How are you thinking of persistence of stats if it was done after the import? After import has a mixture of options. The internally consistent approach is to use the metadata update events already on the event distributor and which are accessible through system procedures. However there is no default backing store yet for that. For none clustered environments it certainly makes sense to just write out the metadata again (if it's supposed to be cached) once the costing update has finished. > Add more options for costing during import > ------------------------------------------ > > Key: TEIID-2790 > URL: https://issues.jboss.org/browse/TEIID-2790 > Project: Teiid > Issue Type: Feature Request > Components: JDBC Connector, Misc. Connectors > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Fix For: 8.7 > > > With many jdbc sources full cardinality is reported from the index info for non statistical indexes (such as mysql). This should be used to set the cardinality of the table. Failing that there should be an option to use an aggregate call or other source stats. > This is not quite the same as TEIID-245 as it's happening at import time, but the two do overlap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2813) Report translator CPU time

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ] Steven Hawkins edited comment on TEIID-2813 at 1/23/14 4:15 PM: ---------------------------------------------------------------- I had to catch up on the associated thread and redo my first comment. The only question that I have to the patch is whether there is ever a time when thread time is supported/enabled yet you wouldn't want it logged - I'm assuming probably not unless as it seems like the overhead would be low. was (Author: shawkins): We do report start/end times so you can derive system clock times. Can you further define CPU time? If not system clock time, that seems like something that is well below our concern and most of the translator time is typically IO bound in blocking calls in any case. > Report translator CPU time > -------------------------- > > Key: TEIID-2813 > URL: https://issues.jboss.org/browse/TEIID-2813 > Project: Teiid > Issue Type: Feature Request > Reporter: Mark Addleman > Assignee: Steven Hawkins > > The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2813) Report translator CPU time

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2813: --------------------------------------- We do report start/end times so you can derive system clock times. Can you further define CPU time? If not system clock time, that seems like something that is well below our concern and most of the translator time is typically IO bound in blocking calls in any case. > Report translator CPU time > -------------------------- > > Key: TEIID-2813 > URL: https://issues.jboss.org/browse/TEIID-2813 > Project: Teiid > Issue Type: Feature Request > Reporter: Mark Addleman > Assignee: Steven Hawkins > > The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2813) Report translator CPU time

by Mark Addleman (JIRA)

Mark Addleman created TEIID-2813: ------------------------------------ Summary: Report translator CPU time Key: TEIID-2813 URL: https://issues.jboss.org/browse/TEIID-2813 Project: Teiid Issue Type: Feature Request Reporter: Mark Addleman Assignee: Steven Hawkins The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2790) Add more options for costing during import

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2790?page=com.atlassian.jira.plugin... ] Ramesh Reddy commented on TEIID-2790: ------------------------------------- I thought we have extension properties for the catalog/schema, if not adding them is good idea on the table. If we go async route, I say we make available for execution through system procedure execution for ad-hoc. How are you thinking of persistence of stats if it was done after the import? Other ways to manually input table stats for runtime would be * web-console * vdb.xml * admin-api > Add more options for costing during import > ------------------------------------------ > > Key: TEIID-2790 > URL: https://issues.jboss.org/browse/TEIID-2790 > Project: Teiid > Issue Type: Feature Request > Components: JDBC Connector, Misc. Connectors > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Fix For: 8.7 > > > With many jdbc sources full cardinality is reported from the index info for non statistical indexes (such as mysql). This should be used to set the cardinality of the table. Failing that there should be an option to use an aggregate call or other source stats. > This is not quite the same as TEIID-245 as it's happening at import time, but the two do overlap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2790) Add more options for costing during import

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2790?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-2790: --------------------------------------- We could add logic to delay/asynchly load stats on import and an option to load stats on startup. The issue with the latter is that in general our metadata is not in a format conducive toward using database specific logic - the name in source is not generally guaranteed to be in a specific format (and we really want it in terms of its name components). If it's not in a format such that the schema name is readily separable, then the best we can do is issue an aggregate query which can be quite expensive. An in between approach would be to add catalog/schema/table name as extension properties on the table at import time to make subsequent logic easier. I also need to look at expanding the applicable sources for stats gathering and adding the column stats logic for oracle. > Add more options for costing during import > ------------------------------------------ > > Key: TEIID-2790 > URL: https://issues.jboss.org/browse/TEIID-2790 > Project: Teiid > Issue Type: Feature Request > Components: JDBC Connector, Misc. Connectors > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Fix For: 8.7 > > > With many jdbc sources full cardinality is reported from the index info for non statistical indexes (such as mysql). This should be used to set the cardinality of the table. Failing that there should be an option to use an aggregate call or other source stats. > This is not quite the same as TEIID-245 as it's happening at import time, but the two do overlap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2790) Add more options for costing during import

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-2790?page=com.atlassian.jira.plugin... ] Work on TEIID-2790 started by Steven Hawkins. > Add more options for costing during import > ------------------------------------------ > > Key: TEIID-2790 > URL: https://issues.jboss.org/browse/TEIID-2790 > Project: Teiid > Issue Type: Feature Request > Components: JDBC Connector, Misc. Connectors > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Fix For: 8.7 > > > With many jdbc sources full cardinality is reported from the index info for non statistical indexes (such as mysql). This should be used to set the cardinality of the table. Failing that there should be an option to use an aggregate call or other source stats. > This is not quite the same as TEIID-245 as it's happening at import time, but the two do overlap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2809) Support for property substitution in vdb.xml files

by Ramesh Reddy (JIRA)

[ https://issues.jboss.org/browse/TEIID-2809?page=com.atlassian.jira.plugin... ] Ramesh Reddy updated TEIID-2809: -------------------------------- Labels: Alpha2 (was: Alpaha2) > Support for property substitution in vdb.xml files > -------------------------------------------------- > > Key: TEIID-2809 > URL: https://issues.jboss.org/browse/TEIID-2809 > Project: Teiid > Issue Type: Enhancement > Affects Versions: 8.5 > Environment: EAP 6.1 Alpha , Teiid 8.5 > Reporter: Madhu Garimilla > Assignee: Ramesh Reddy > Labels: Alpha2 > Fix For: 8.7 > > > I have created a vdb.xml where i have declared my translators in it. I would like to pass some parameters to one of my translators so that i do not need to change my vdb.xml whenever there is a change in value. These parameters derive the values from the system properties. So, i am using ${} notation to pass the value of the system properties to my translator. When i try to deploy my VDB and try to access the parameters inside the translator, These system properties are not being resolved to the actual values. > As per the thread https://community.jboss.org/thread/236325, this is not being supported as of now and requires an enhancement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

10 years, 11 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues January 2014