[teiid-issues] [JBoss JIRA] (TEIID-2842) Impala/ HIVE2: Unsupported compress file

[Ramesh Reddy (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12944815#comment-12944815 ] 

Ramesh Reddy commented on TEIID-2842:
-------------------------------------

I downloaded the Cloudera's Quick Start VM instance for "Virtual Box", followed the your instructions on the  "Import Data Into Hadoop" document, and used the DDL from hiveddl.txt ( I changed the  file location to point to the right directory /home/cloudera/ominutelogs) where the .gz file as

{code}
CREATE EXTERNAL TABLE omniturelogs
(
col_1 string, col_2 string, col_3 string, col_4 string, col_5 string, col_6 string, col_7 string, col_8 string,  ....col_178 string
) 
-- PARTITIONED BY (id string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE 
LOCATION "/user/coludera/omniturelogs";
{code}

Then I can see I can issue 
{code}
DESCRIBE omniturelogs;
DESCRIBE FORMATTED omniturelogs;
{code}

with out any issues. I started a Hive2 server
{code}
hive --service hiveserver2
{code}

then connect to the hive server using hive driver from my local machine over JDBC, then issued same describe commands, they worked fine too.

So, given that do you think there may be some configuration option you may be missing in your setup?
                
> Impala/ HIVE2:  Unsupported compress file
> -----------------------------------------
>
>                 Key: TEIID-2842
>                 URL: https://issues.jboss.org/browse/TEIID-2842
>             Project: Teiid
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: 8.6
>         Environment: Impala/HIVE2 with Teiid
>            Reporter: Ivan Chan
>            Assignee: Ramesh Reddy
>              Labels: teiid
>             Fix For: 8.7
>
>
> Teiid makes a query call "Describe table" in HiveMetadataProcessor.addTable(....).   However, if the table contains compress file, HIVE2 driver would return the following execption:
> {code}
> java.sql.SQLException: AnalysisException: Failed to load metadata for table: default.omniturelogs
> CAUSED BY: TableLoadingException: Failed to load metadata for table: 
> omniturelogs
> CAUSED BY: RuntimeException: Compressed text files are not supported: hdfs://localhost.localdomain:8020/user/cloudera/omniturelogs/Omniture.0.tsv.gz
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:161)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:149)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:182)
> at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:246)	
> at org.apache.tomcat.dbcp.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at org.apache.tomcat.dbcp.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)	
> at org.teiid.translator.hive.HiveMetadataProcessor.addTable(HiveMetadataProcessor.java:112)	
> at org.teiid.translator.hive.HiveMetadataProcessor.getConnectorMetadata(HiveMetadataProcessor.java:48)
> 	
> at org.teiid.translator.jdbc.JDBCExecutionFactory.getMetadata(JDBCExecutionFactory.java:311)
> 	... 155 more
> {code}
> Maybe Teiid should call "DESCRIBE FORMATTED table" for table that contains compress file.
> Reference:
> http://grokbase.com/t/cloudera/impala-user/13b7c9apmx/impala-failed-to-query-snappy-compression-data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira