[JBoss JIRA] (TEIID-3153) Allow to write stream data with large size (>10G)
by Haifen Bi (JIRA)
Haifen Bi created TEIID-3153:
--------------------------------
Summary: Allow to write stream data with large size (>10G)
Key: TEIID-3153
URL: https://issues.jboss.org/browse/TEIID-3153
Project: Teiid
Issue Type: Enhancement
Reporter: Haifen Bi
Assignee: Steven Hawkins
We have quite large size of data file (>10G) that needs to be sent back to server from client. Following is JDBC client code:
void writeData(int id) {
String sql = "UPDATE DataSet set data = ? WHERE id = ?";
try {
InputStream is = new FileInputStream("c:\\temp\\data");
this.ps = connection.prepareStatement(sql);
this.ps.setBlob(1, is);
this.ps.setInt(2, id);
ps.executeUpdate();
} catch (Exception e) {
..
}
}
Teiid throws "lob too big" error when executing above code:
10:01:18,801 WARN [org.teiid.TRANSPORT] (New I/O worker #1) TEIID40113 Unhandled exception, aborting operation: java.io.StreamCorruptedException: lob too big:
429496456065534 (max: 4294967296)
at org.teiid.transport.ObjectDecoder.decode(ObjectDecoder.java:142) [teiid-runtime-8.7.0.Final.jar:8.7.0.Final]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) [netty-3.6.2.Final.jar:]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.2.Final.jar:]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.2.Final.jar:]
It seemed teiid has stream data size limit (4G?). With this limitation, it is not possible to send more than 4G of stream data in Teiid.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3142) Vanilla Hive sorting issue
by Filip Nguyen (JIRA)
[ https://issues.jboss.org/browse/TEIID-3142?page=com.atlassian.jira.plugin... ]
Filip Nguyen commented on TEIID-3142:
-------------------------------------
I couldn't find any setting of collation. Also by googling I found a mailing list entry that says the Hive may not support any collation setting:
http://qnalist.com/questions/950113/does-hive-support-collation
Do you think that the sensible default is to turn off orderby? I guess we still want the ordering to be consistent with other translator, even with character data.
> Vanilla Hive sorting issue
> --------------------------
>
> Key: TEIID-3142
> URL: https://issues.jboss.org/browse/TEIID-3142
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.7.1
> Reporter: Filip Nguyen
> Assignee: Steven Hawkins
>
> It seems that Vanilla Apache Hive 0.13 sorts in a yet another different way than what we expect and also differently from Cloudera Imapla (another Hive flavor).
> SELECT BQT1.SmallA.StringNum FROM BQT1.SmallA order by BQT1.SmallA.StringNum
> 1: -1
> 2: -10
> 3: -11
> 4: -12
> 5: -13
> 6: -14
> 7: -15
> 8: -16
> 9: -17
> 10: -18
> 11: -19
> 12: -2
> 13: -20
> 14: -21
> 15: -22
> 16: -24
> 17: -3
> 18: -4
> 19: -5
> 20: -6
> 21: -8
> 22: -9
> 23: 0
> 24: 1
> 25: 10
> 26: 11
> 27: 12
> 28: 13
> 29: 14
> 30: 15
> 31: 16
> 32: 17
> 33: 18
> 34: 19
> 35: 2
> 36: 20
> 37: 21
> 38: 22
> 39: 23
> 40: 24
> 41: 3
> 42: 4
> 43: 5
> 44: 6
> 45: 7
> 46: 8
> 47: null
> 48: null
> 49: null
> 50: null
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3134) Impala sorting Issue
by Filip Nguyen (JIRA)
[ https://issues.jboss.org/browse/TEIID-3134?page=com.atlassian.jira.plugin... ]
Filip Nguyen commented on TEIID-3134:
-------------------------------------
It seems that Impala doesnt support collation
http://www.cloudera.com/content/cloudera/en/documentation/cloudera-impala...
So the sensible default for impala translator would be to turn off order by support.
> Impala sorting Issue
> --------------------
>
> Key: TEIID-3134
> URL: https://issues.jboss.org/browse/TEIID-3134
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.7.1
> Reporter: Van Halbert
>
> String sorting of impala is inconsistent with our expectations:
> Impala sorting
> ---------------------------
> SELECT BQT1.SmallA.StringNum FROM BQT1.SmallA
> Results
> 1: -1
> 2: -10
> 3: -11
> 4: -12
> 5: -13
> 6: -14
> 7: -15
> 8: -16
> 9: -17
> 10: -18
> 11: -19
> 12: -2
> 13: -20
> 14: -21
> 15: -22
> 16: -24
> 17: -3
> 18: -4
> 19: -5
> 20: -6
> 21: -8
> 22: -9
> 23: 0
> 24: 1
> 25: 10
> 26: 11
> 27: 12
> 28: 13
> 29: 14
> 30: 15
> 31: 16
> 32: 17
> 33: 18
> 34: 19
> 35: 2
> 36: 20
> 37: 21
> 38: 22
> 39: 23
> 40: 24
> 41: 3
> 42: 4
> 43: 5
> 44: 6
> 45: 7
> 46: 8
> 47: null
> 48: null
> 49: null
> 50: null
> Expected
> ---------------------------
> While we expect sorting to be consistent with other sources:
> 1: 0
> 2: 1
> 3: -1
> 4: 10
> 5: -10
> 6: 11
> 7: -11
> 8: 12
> 9: -12
> 10: 13
> 11: -13
> 12: 14
> 13: -14
> 14: 15
> 15: -15
> 16: 16
> 17: -16
> 18: 17
> 19: -17
> 20: 18
> 21: -18
> 22: 19
> 23: -19
> 24: 2
> 25: -2
> 26: 20
> 27: -20
> 28: 21
> 29: -21
> 30: 22
> 31: -22
> 32: 23
> 33: 24
> 34: -24
> 35: 3
> 36: -3
> 37: 4
> 38: -4
> 39: 5
> 40: -5
> 41: 6
> 42: -6
> 43: 7
> 44: 8
> 45: -8
> 46: -9
> 47: null
> 48: null
> 49: null
> 50: null
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3141) Bad result of query with GROUB BY clause (underlying sybase15 datasource)
by Juraj Duráni (JIRA)
[ https://issues.jboss.org/browse/TEIID-3141?page=com.atlassian.jira.plugin... ]
Juraj Duráni commented on TEIID-3141:
-------------------------------------
One more note: there is similar problem with functions AVG, ASIN, ACOS,... (returns always integer instead real).
> Bad result of query with GROUB BY clause (underlying sybase15 datasource)
> -------------------------------------------------------------------------
>
> Key: TEIID-3141
> URL: https://issues.jboss.org/browse/TEIID-3141
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 8.7.1
> Environment: OS: fedora20
> arch: x86_64
> java: sun 1.7
> Reporter: Juraj Duráni
> Assignee: Steven Hawkins
>
> Description:
> There is sybase15 database with table named SmallA and table named SmallA in VDB which is mapped to sybase table (see tables definition below).
> I am trying to run query against VDB:
> SELECT
> INTKEY, STRINGKEY, INTNUM, STRINGNUM, FLOATNUM, LONGNUM, DOUBLENUM, BYTENUM, DATEVALUE, TIMEVALUE, TIMESTAMPVALUE, BOOLEANVALUE, CHARVALUE, SHORTVALUE, BIGINTEGERVALUE, BIGDECIMALVALUE
> FROM
> BQT1.SMALLA
> GROUP BY
> INTKEY, STRINGKEY, INTNUM, STRINGNUM, FLOATNUM, LONGNUM, DOUBLENUM, BYTENUM, DATEVALUE, TIMEVALUE, TIMESTAMPVALUE, BOOLEANVALUE, CHARVALUE, SHORTVALUE, BIGINTEGERVALUE, BIGDECIMALVALUE
> Result is table which misses some values (the other values are OK):
> FloatNum: always 0
> ByteNum: always 0
> DateValue: always 1900-01-01
> TimeValue: always 00:00:00
> BooleanValue: always 'false'
> CharValue: always empty character
> ShortValue: always 0
> BigIntegerValue: always 0
> BigDecimalValue: always 0
> After removing 'INTKEY' and 'STRINGKEY' from the query is result OK (sybase15 has indices only for these two columns).
> ///////////////////
> Table definition
> ///////////////////
> SmallA (sybase) has these columns (name:type):
> IntKey:int -> PRIMARY KEY, HAS INDEX
> StringKey:varchar -> HAS INDEX
> IntNum:int
> StringNum:varchar
> FloatNum:float
> LongNum:numeric
> DoubleNum:float
> ByteNum:real
> DateValue:datetime
> TimeValue:datetime
> TimestampValue:datetime
> BooleanValue:tinyint
> CharValue:char
> ShortValue:numeric
> BigIntegerValue:numeric
> BigDecimalValue:numeric
> ObjectValue:text
> SmallA (VDB) has these columns (name:type):
> IntKey:integer
> StringKey:string
> IntNum:integer
> StringNum:string
> FloatNum:float
> LongNum:long
> DoubleNum:double
> ByteNum:byte
> DateValue:date
> TimeValue:time
> TimestampValue:timestamp
> BooleanValue:boolean
> CharValue:char
> ShortValue:short
> BigIntegerValue:biginteger
> BigDecimalValue:bigdecimal
> ObjectValue:object
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3152) result set close after session close reported as an error
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3152?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-3152.
-----------------------------------
Resolution: Done
Checked for statement close prior to the client closing. Also corrected the exception conversion logic so that we won't convert exceptions to more severe forms and added logic to filter out netty exceptions related to clients closing the connection. Apparently it's a general issue that netty will notify listeners before closing the channel.
> result set close after session close reported as an error
> ---------------------------------------------------------
>
> Key: TEIID-3152
> URL: https://issues.jboss.org/browse/TEIID-3152
> Project: Teiid
> Issue Type: Bug
> Components: Server
> Affects Versions: 8.7
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.7.1, 8.9
>
>
> An invalid session exception will be recorded as an error rather than a warning. This is due to the exceptionutil converting inappropriately converting the exception into a teiid component exception.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3133) Impala Time/Date/Timestamp handling
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3133?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3133:
---------------------------------------
No that should be supported. It looks like both the Hive and the Impala translator are marked as supporting all forms of the cast/convert function, which can result in attempting to retrieve a type that is not supported. This will require a code change to the supportsConvert function to narrow to an appropriate set of conversions.
> Impala Time/Date/Timestamp handling
> -----------------------------------
>
> Key: TEIID-3133
> URL: https://issues.jboss.org/browse/TEIID-3133
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.7.1
> Reporter: Van Halbert
>
> Impala supports only TIMESTAMP datatype (see here http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/lates...)
> Converted everything into timestamp and now I am trying to get Date and Time parts of the timestamp in Teiid e.g.
> SELECT DISTINCT convert(Source.SmallA.TimeValue, time) as TimeValue FROM Source.SmallA ORDER BY TimeValue
> But I get TranslatorException: Unexpected exception while translating results: Method not supported
> Tried to workaround this but with limited success. For Date it is possible to use to_date function that returns string reprsentation of the date part. For Time I wasn't able to find the solution (have not yet figured out how to use extract function through Teiid).
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/lates...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3142) Vanilla Hive sorting issue
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3142?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3142:
---------------------------------------
Yes, it can all be disabled. Which is not quite desirable if this a collation issue only affecting character data. See the translator property supportsOrderBy.
> Vanilla Hive sorting issue
> --------------------------
>
> Key: TEIID-3142
> URL: https://issues.jboss.org/browse/TEIID-3142
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.7.1
> Reporter: Filip Nguyen
> Assignee: Steven Hawkins
>
> It seems that Vanilla Apache Hive 0.13 sorts in a yet another different way than what we expect and also differently from Cloudera Imapla (another Hive flavor).
> SELECT BQT1.SmallA.StringNum FROM BQT1.SmallA order by BQT1.SmallA.StringNum
> 1: -1
> 2: -10
> 3: -11
> 4: -12
> 5: -13
> 6: -14
> 7: -15
> 8: -16
> 9: -17
> 10: -18
> 11: -19
> 12: -2
> 13: -20
> 14: -21
> 15: -22
> 16: -24
> 17: -3
> 18: -4
> 19: -5
> 20: -6
> 21: -8
> 22: -9
> 23: 0
> 24: 1
> 25: 10
> 26: 11
> 27: 12
> 28: 13
> 29: 14
> 30: 15
> 31: 16
> 32: 17
> 33: 18
> 34: 19
> 35: 2
> 36: 20
> 37: 21
> 38: 22
> 39: 23
> 40: 24
> 41: 3
> 42: 4
> 43: 5
> 44: 6
> 45: 7
> 46: 8
> 47: null
> 48: null
> 49: null
> 50: null
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3142) Vanilla Hive sorting issue
by Filip Nguyen (JIRA)
[ https://issues.jboss.org/browse/TEIID-3142?page=com.atlassian.jira.plugin... ]
Filip Nguyen commented on TEIID-3142:
-------------------------------------
Steven, is there a possibility for a user to override the translator's support for order by?
> Vanilla Hive sorting issue
> --------------------------
>
> Key: TEIID-3142
> URL: https://issues.jboss.org/browse/TEIID-3142
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.7.1
> Reporter: Filip Nguyen
> Assignee: Steven Hawkins
>
> It seems that Vanilla Apache Hive 0.13 sorts in a yet another different way than what we expect and also differently from Cloudera Imapla (another Hive flavor).
> SELECT BQT1.SmallA.StringNum FROM BQT1.SmallA order by BQT1.SmallA.StringNum
> 1: -1
> 2: -10
> 3: -11
> 4: -12
> 5: -13
> 6: -14
> 7: -15
> 8: -16
> 9: -17
> 10: -18
> 11: -19
> 12: -2
> 13: -20
> 14: -21
> 15: -22
> 16: -24
> 17: -3
> 18: -4
> 19: -5
> 20: -6
> 21: -8
> 22: -9
> 23: 0
> 24: 1
> 25: 10
> 26: 11
> 27: 12
> 28: 13
> 29: 14
> 30: 15
> 31: 16
> 32: 17
> 33: 18
> 34: 19
> 35: 2
> 36: 20
> 37: 21
> 38: 22
> 39: 23
> 40: 24
> 41: 3
> 42: 4
> 43: 5
> 44: 6
> 45: 7
> 46: 8
> 47: null
> 48: null
> 49: null
> 50: null
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months
[JBoss JIRA] (TEIID-3142) Vanilla Hive sorting issue
by Filip Nguyen (JIRA)
[ https://issues.jboss.org/browse/TEIID-3142?page=com.atlassian.jira.plugin... ]
Filip Nguyen commented on TEIID-3142:
-------------------------------------
Thanks Steven, I will first try to configure the server to get desired sorting.
> Vanilla Hive sorting issue
> --------------------------
>
> Key: TEIID-3142
> URL: https://issues.jboss.org/browse/TEIID-3142
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.7.1
> Reporter: Filip Nguyen
> Assignee: Steven Hawkins
>
> It seems that Vanilla Apache Hive 0.13 sorts in a yet another different way than what we expect and also differently from Cloudera Imapla (another Hive flavor).
> SELECT BQT1.SmallA.StringNum FROM BQT1.SmallA order by BQT1.SmallA.StringNum
> 1: -1
> 2: -10
> 3: -11
> 4: -12
> 5: -13
> 6: -14
> 7: -15
> 8: -16
> 9: -17
> 10: -18
> 11: -19
> 12: -2
> 13: -20
> 14: -21
> 15: -22
> 16: -24
> 17: -3
> 18: -4
> 19: -5
> 20: -6
> 21: -8
> 22: -9
> 23: 0
> 24: 1
> 25: 10
> 26: 11
> 27: 12
> 28: 13
> 29: 14
> 30: 15
> 31: 16
> 32: 17
> 33: 18
> 34: 19
> 35: 2
> 36: 20
> 37: 21
> 38: 22
> 39: 23
> 40: 24
> 41: 3
> 42: 4
> 43: 5
> 44: 6
> 45: 7
> 46: 8
> 47: null
> 48: null
> 49: null
> 50: null
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
9 years, 7 months