[teiid-issues] [JBoss JIRA] (TEIID-2383) InputStreamReader throws MalformedInputException when handling multi-byte characters

[ https://issues.jboss.org/browse/TEIID-2383?page=com.atlassian.jira.plugin... ] Hisanobu Okuda updated TEIID-2383: ---------------------------------- Steps to Reproduce: 1. unzip sample.zip 2. copy files in sample.zip {code} sample |-- datafile -> $JBOSS_HOME/servers/$profile | `-- postal_utf8.csv |-- file-test-vdb.xml -> $JBOSS_HOME/servers/$profile/deploy `-- test-ds.xml -> $JBOSS_HOME/servers/$profile/deploy {code} 3. run simple client ex) {code} ./run.sh {code} 4. After 256 raws fetched, MalformedInputException occurs as follow:- {code} 11:42:16,219 DEBUG [PROCESSOR] Request Thread x8XReh9a7kIY.0 - error occurred org.teiid.core.TeiidProcessingException: Input length = 1 at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:267) at org.teiid.query.processor.relational.TextTableNode.readLine(TextTableNode.java:207) at org.teiid.query.processor.relational.TextTableNode.nextBatchDirect(TextTableNode.java:166) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70) at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69) at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:82) at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91) at org.teiid.query.processor.relational.NestedTableJoinStrategy.process(NestedTableJoinStrategy.java:120) at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:199) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:155) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176) at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139) at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105) at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147) at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375) at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288) at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49) at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216) at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:240) at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122) at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.nio.charset.MalformedInputException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:260) at org.teiid.core.util.InputStreamReader.read(InputStreamReader.java:84) at java.io.BufferedReader.fill(BufferedReader.java:136) at java.io.BufferedReader.read(BufferedReader.java:157) at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:245) ... 26 more {code} was: 1. unzip sample.zip 2. copy files in sample.zip sample |-- datafile -> $JBOSS_HOME/servers/$profile | `-- postal_utf8.csv |-- file-test-vdb.xml -> $JBOSS_HOME/servers/$profile/deploy `-- test-ds.xml -> $JBOSS_HOME/servers/$profile/deploy 3. run simple client ex) ./run.sh localhost 31000 DynamicPortfolio "select test.* from (call TestData.getTextFiles('postal_utf8.csv')) f, TEXTTABLE(f.file COLUMNS id string, Town1 string, City1 string, State1 string HEADER) test" 4. After 256 raws fetched, MalformedInputException occurs as follow:- 11:42:16,219 DEBUG [PROCESSOR] Request Thread x8XReh9a7kIY.0 - error occurred org.teiid.core.TeiidProcessingException: Input length = 1 at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:267) at org.teiid.query.processor.relational.TextTableNode.readLine(TextTableNode.java:207) at org.teiid.query.processor.relational.TextTableNode.nextBatchDirect(TextTableNode.java:166) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70) at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69) at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:82) at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91) at org.teiid.query.processor.relational.NestedTableJoinStrategy.process(NestedTableJoinStrategy.java:120) at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:199) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:155) at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280) at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176) at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139) at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105) at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147) at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375) at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288) at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49) at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216) at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:240) at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122) at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.nio.charset.MalformedInputException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:260) at org.teiid.core.util.InputStreamReader.read(InputStreamReader.java:84) at java.io.BufferedReader.fill(BufferedReader.java:136) at java.io.BufferedReader.read(BufferedReader.java:157) at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:245) ... 26 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira