[JBoss JIRA] (TEIID-2814) HIVE: quote is removed for TIMESTAMP field in where clause
by Ivan Chan (JIRA)
Ivan Chan created TEIID-2814:
--------------------------------
Summary: HIVE: quote is removed for TIMESTAMP field in where clause
Key: TEIID-2814
URL: https://issues.jboss.org/browse/TEIID-2814
Project: Teiid
Issue Type: Bug
Components: Query Engine
Affects Versions: 8.6
Environment: HIVE and Teiid 8.6
Reporter: Ivan Chan
Assignee: Steven Hawkins
I am running teiid 8.6 against HIVE. And I found out that quote was removed for TIMESTAMP field in where clause. And it caused java.sql.SQLException: Error while processing statement: FAILED: ParseException line 1:88 missing EOF
Original SQL:
select "the_date",
"store_sales"
from "hive2"."sales_seq"
where ("the_date" > '1970-05-16 00:10:04')
Teiid generated SQL:
SELECT g_0.the_date, g_0.store_sales FROM sales_seq g_0 WHERE g_0.the_date > 1970-05-16 00:10:04.0
Exception:
Caused by: java.sql.SQLException: Error while processing statement: FAILED:
ParseException line 1:88 missing EOF at '00' near '16'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (TEIID-2813) Report translator CPU time
by Mark Addleman (JIRA)
[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ]
Mark Addleman commented on TEIID-2813:
--------------------------------------
In my experience, the overhead of grabbing CPU time is a little higher than getting clock time but the only time this matters is in tight-loop situations. Since the changes apply to fairly heavy-weight methods anyway (particularly, handleBatch() versus next()), I don't think overhead is much of an issue. If you want, I can introduce some switch.
> Report translator CPU time
> --------------------------
>
> Key: TEIID-2813
> URL: https://issues.jboss.org/browse/TEIID-2813
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Mark Addleman
> Assignee: Steven Hawkins
>
> The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (TEIID-2813) Report translator CPU time
by Mark Addleman (JIRA)
[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ]
Mark Addleman commented on TEIID-2813:
--------------------------------------
There is a small discussion of this at https://community.jboss.org/thread/236413 I want to know the accumulated CPU from the translator's execute() and next() calls. I agree that nearly all translators are IO bound but some of our translators take on non-trivial CPU work to re-orient data that comes out of the source. In order to better direct our tuning efforts, having CPU time reported against the translator is useful.
Ideally, Teiid would also report the total CPU required to process a user query so we can make query tuning decisions and, perhaps, supports* decisions. It is possible that even though a source can handle filtering, Teiid may be able to process the filter more efficiently - we have some very odd data sources. At this time, we don't *need* CPU time at the user-query level so this JIRA only addresses the translator. I was planning on reviewing the code to see how difficult it would be to add instrumentation at the higher level
> Report translator CPU time
> --------------------------
>
> Key: TEIID-2813
> URL: https://issues.jboss.org/browse/TEIID-2813
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Mark Addleman
> Assignee: Steven Hawkins
>
> The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (TEIID-2790) Add more options for costing during import
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2790?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2790:
---------------------------------------
> I thought we have extension properties for the catalog/schema, if not adding them is good idea on the table.
No, nor would any hand created metadata / designer vdb contain them.
> I say we make available for execution through system procedure execution for ad-hoc
Yes, that is where this issue dove tails with TEIID-245
> How are you thinking of persistence of stats if it was done after the import?
After import has a mixture of options. The internally consistent approach is to use the metadata update events already on the event distributor and which are accessible through system procedures. However there is no default backing store yet for that.
For none clustered environments it certainly makes sense to just write out the metadata again (if it's supposed to be cached) once the costing update has finished.
> Add more options for costing during import
> ------------------------------------------
>
> Key: TEIID-2790
> URL: https://issues.jboss.org/browse/TEIID-2790
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector, Misc. Connectors
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.7
>
>
> With many jdbc sources full cardinality is reported from the index info for non statistical indexes (such as mysql). This should be used to set the cardinality of the table. Failing that there should be an option to use an aggregate call or other source stats.
> This is not quite the same as TEIID-245 as it's happening at import time, but the two do overlap.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (TEIID-2813) Report translator CPU time
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ]
Steven Hawkins edited comment on TEIID-2813 at 1/23/14 4:15 PM:
----------------------------------------------------------------
I had to catch up on the associated thread and redo my first comment. The only question that I have to the patch is whether there is ever a time when thread time is supported/enabled yet you wouldn't want it logged - I'm assuming probably not unless as it seems like the overhead would be low.
was (Author: shawkins):
We do report start/end times so you can derive system clock times. Can you further define CPU time? If not system clock time, that seems like something that is well below our concern and most of the translator time is typically IO bound in blocking calls in any case.
> Report translator CPU time
> --------------------------
>
> Key: TEIID-2813
> URL: https://issues.jboss.org/browse/TEIID-2813
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Mark Addleman
> Assignee: Steven Hawkins
>
> The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (TEIID-2813) Report translator CPU time
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2813?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2813:
---------------------------------------
We do report start/end times so you can derive system clock times. Can you further define CPU time? If not system clock time, that seems like something that is well below our concern and most of the translator time is typically IO bound in blocking calls in any case.
> Report translator CPU time
> --------------------------
>
> Key: TEIID-2813
> URL: https://issues.jboss.org/browse/TEIID-2813
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Mark Addleman
> Assignee: Steven Hawkins
>
> The COMMAND_LOG ought to report the CPU time accumulated by the translator for performance tuning purposes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (TEIID-2790) Add more options for costing during import
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2790?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2790:
-------------------------------------
I thought we have extension properties for the catalog/schema, if not adding them is good idea on the table. If we go async route, I say we make available for execution through system procedure execution for ad-hoc. How are you thinking of persistence of stats if it was done after the import?
Other ways to manually input table stats for runtime would be
* web-console
* vdb.xml
* admin-api
> Add more options for costing during import
> ------------------------------------------
>
> Key: TEIID-2790
> URL: https://issues.jboss.org/browse/TEIID-2790
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector, Misc. Connectors
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.7
>
>
> With many jdbc sources full cardinality is reported from the index info for non statistical indexes (such as mysql). This should be used to set the cardinality of the table. Failing that there should be an option to use an aggregate call or other source stats.
> This is not quite the same as TEIID-245 as it's happening at import time, but the two do overlap.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (TEIID-2790) Add more options for costing during import
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2790?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2790:
---------------------------------------
We could add logic to delay/asynchly load stats on import and an option to load stats on startup. The issue with the latter is that in general our metadata is not in a format conducive toward using database specific logic - the name in source is not generally guaranteed to be in a specific format (and we really want it in terms of its name components). If it's not in a format such that the schema name is readily separable, then the best we can do is issue an aggregate query which can be quite expensive. An in between approach would be to add catalog/schema/table name as extension properties on the table at import time to make subsequent logic easier.
I also need to look at expanding the applicable sources for stats gathering and adding the column stats logic for oracle.
> Add more options for costing during import
> ------------------------------------------
>
> Key: TEIID-2790
> URL: https://issues.jboss.org/browse/TEIID-2790
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector, Misc. Connectors
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.7
>
>
> With many jdbc sources full cardinality is reported from the index info for non statistical indexes (such as mysql). This should be used to set the cardinality of the table. Failing that there should be an option to use an aggregate call or other source stats.
> This is not quite the same as TEIID-245 as it's happening at import time, but the two do overlap.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months