[teiid-issues] [JBoss JIRA] (TEIID-2383) InputStreamReader throws MalformedInputException when handling multi-byte characters

Hisanobu Okuda (JIRA) jira-events at lists.jboss.org
Fri Feb 8 01:33:51 EST 2013


     [ https://issues.jboss.org/browse/TEIID-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hisanobu Okuda updated TEIID-2383:
----------------------------------

    Steps to Reproduce: 
1. unzip sample.zip
2. copy files in sample.zip

{code}
sample 
|-- datafile  -> $JBOSS_HOME/servers/$profile
|   `-- postal_utf8.csv
|-- file-test-vdb.xml -> $JBOSS_HOME/servers/$profile/deploy
`-- test-ds.xml       -> $JBOSS_HOME/servers/$profile/deploy
{code}

3. run simple client
ex)

{code}
./run.sh
{code}

4. After 256 raws fetched, MalformedInputException occurs as follow:-

{code}
11:42:16,219 DEBUG [PROCESSOR] Request Thread x8XReh9a7kIY.0 - error occurred
org.teiid.core.TeiidProcessingException: Input length = 1
	at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:267)
	at org.teiid.query.processor.relational.TextTableNode.readLine(TextTableNode.java:207)
	at org.teiid.query.processor.relational.TextTableNode.nextBatchDirect(TextTableNode.java:166)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70)
	at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69)
	at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:82)
	at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91)
	at org.teiid.query.processor.relational.NestedTableJoinStrategy.process(NestedTableJoinStrategy.java:120)
	at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:199)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:155)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176)
	at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139)
	at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105)
	at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147)
	at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375)
	at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288)
	at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49)
	at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216)
	at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:240)
	at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122)
	at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
	at org.teiid.core.util.InputStreamReader.read(InputStreamReader.java:84)
	at java.io.BufferedReader.fill(BufferedReader.java:136)
	at java.io.BufferedReader.read(BufferedReader.java:157)
	at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:245)
	... 26 more
{code}

  was:
1. unzip sample.zip
2. copy files in sample.zip

sample 
|-- datafile  -> $JBOSS_HOME/servers/$profile
|   `-- postal_utf8.csv
|-- file-test-vdb.xml -> $JBOSS_HOME/servers/$profile/deploy
`-- test-ds.xml       -> $JBOSS_HOME/servers/$profile/deploy

3. run simple client
ex)
./run.sh localhost 31000 DynamicPortfolio "select test.* from (call TestData.getTextFiles('postal_utf8.csv')) f, TEXTTABLE(f.file COLUMNS id string, Town1 string, City1 string, State1 string HEADER) test"

4. After 256 raws fetched, MalformedInputException occurs as follow:-

11:42:16,219 DEBUG [PROCESSOR] Request Thread x8XReh9a7kIY.0 - error occurred
org.teiid.core.TeiidProcessingException: Input length = 1
	at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:267)
	at org.teiid.query.processor.relational.TextTableNode.readLine(TextTableNode.java:207)
	at org.teiid.query.processor.relational.TextTableNode.nextBatchDirect(TextTableNode.java:166)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:70)
	at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:69)
	at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:82)
	at org.teiid.common.buffer.AbstractTupleSource.hasNext(AbstractTupleSource.java:91)
	at org.teiid.query.processor.relational.NestedTableJoinStrategy.process(NestedTableJoinStrategy.java:120)
	at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:199)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:155)
	at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:280)
	at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:176)
	at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:139)
	at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:105)
	at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:147)
	at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:375)
	at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:288)
	at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49)
	at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216)
	at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:240)
	at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122)
	at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
	at org.teiid.core.util.InputStreamReader.read(InputStreamReader.java:84)
	at java.io.BufferedReader.fill(BufferedReader.java:136)
	at java.io.BufferedReader.read(BufferedReader.java:157)
	at org.teiid.query.processor.relational.TextTableNode.readChar(TextTableNode.java:245)
	... 26 more



    
> InputStreamReader throws MalformedInputException when handling multi-byte characters
> ------------------------------------------------------------------------------------
>
>                 Key: TEIID-2383
>                 URL: https://issues.jboss.org/browse/TEIID-2383
>             Project: Teiid
>          Issue Type: Bug
>          Components: Common
>    Affects Versions: 7.7.1
>            Reporter: Hisanobu Okuda
>            Assignee: Steven Hawkins
>
> My VDB uses File translator to handle a csv file which contains multi-byte characters. Reading some csv file via the VDB, MalformedInputException always occurs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


More information about the teiid-issues mailing list