[teiid-issues] [JBoss JIRA] (TEIID-2668) Teiid procedure input parameter has limit for bulk data

[Steven Hawkins (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12812825#comment-12812825 ] 

Steven Hawkins commented on TEIID-2668:
---------------------------------------

With the 8.4 fix to the client side you are more than likely hitting the max object size limitation on message packets on the server side.  I would have expected to see a log to that effect:

data length too big: 1144595 (max: 1048576)

This protects the server from OOM in case clients attempt to flood the server with data.  

If this is the case there are a couple of considerations:

That limit is hard-coded at 1 MB.  With the rationale being that large objects would be sent as blobs/clobs/sqlxml which are part of a streaming protocol and not in the request message.  I can hit this limit in your list of double list scenario with 100000+ doubles.

In your case if it is inconvenient to send your objects as blobs, then you would want to increase that value, which would take an enhancement.

In 8.5 using a multidimensional Double array would help somewhat with the enhanced array support as the serialization would be much more compact.  But you would still have an issue with arbitrary objects that are sent as non-lob parameter values.


                
> Teiid procedure input parameter has limit for bulk data
> -------------------------------------------------------
>
>                 Key: TEIID-2668
>                 URL: https://issues.jboss.org/browse/TEIID-2668
>             Project: Teiid
>          Issue Type: Bug
>    Affects Versions: 8.3
>         Environment: Teiid 8.3.
>            Reporter: Jing Li
>            Assignee: Steven Hawkins
>
> We wrote a Teiid procedure, one of the input parameters is List<List<Double>>. We found if we give too many data in there, e.g. 200x375, Teiid will throw out exception before go to my translator:
> org.teiid.jdbc.TeiidSQLException: Error Code:TEIID20013 Message:Error Code:TEIID20013 Message:java.net.SocketException: Connection reset by peer: socket write error
> 	at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:113)
> 	at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:70)
> 	at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:631)
> 	at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:559)
> 	at org.teiid.jdbc.PreparedStatementImpl.execute(PreparedStatementImpl.java:201)
> 	at MyTest.testHorizon3DWrite(MyTest.java:102)
> 	at MyTest.main(MyTest.java:27)
> Caused by: [TeiidComponentException] TEIID20013: Error Code:TEIID20013 Message:Error Code:TEIID20013 Message:java.net.SocketException: Connection reset by peer: socket write error
> 1 [SingleInstanceCommunicationException] TEIID20013: Error Code:TEIID20013 Message:java.net.SocketException: Connection reset by peer: socket write error
> 2 [ExecutionException]java.net.SocketException: Connection reset by peer: socket write error
> 3 [SocketException]Connection reset by peer: socket write error
> 	at org.teiid.client.util.ExceptionUtil.convertException(ExceptionUtil.java:61)
> 	at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler.invoke(SocketServerInstanceImpl.java:374)
> 	at org.teiid.net.socket.SocketServerConnection$1.invoke(SocketServerConnection.java:243)
> 	at $Proxy7.executeRequest(Unknown Source)
> 	at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:629)
> 	... 4 more
> Caused by: [SingleInstanceCommunicationException] TEIID20013: Error Code:TEIID20013 Message:java.net.SocketException: Connection reset by peer: socket write error
> 1 [ExecutionException]java.net.SocketException: Connection reset by peer: socket write error
> 2 [SocketException]Connection reset by peer: socket write error
> 	at org.teiid.net.socket.SocketServerInstanceImpl.send(SocketServerInstanceImpl.java:177)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:102)
> 	at $Proxy1.send(Unknown Source)
> 	at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler.invoke(SocketServerInstanceImpl.java:362)
> 	... 7 more
> Caused by: java.util.concurrent.ExecutionException: java.net.SocketException: Connection reset by peer: socket write error
> 	at org.teiid.client.util.ResultsFuture.convertResult(ResultsFuture.java:100)
> 	at org.teiid.client.util.ResultsFuture.get(ResultsFuture.java:95)
> 	at org.teiid.net.socket.SocketServerInstanceImpl.send(SocketServerInstanceImpl.java:174)
> 	... 14 more
> Caused by: java.net.SocketException: Connection reset by peer: socket write error
> 	at java.net.SocketOutputStream.socketWrite0(Native Method)
> 	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> 	at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
> 	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> 	at java.io.DataOutputStream.write(DataOutputStream.java:90)
> 	at org.teiid.netty.handler.codec.serialization.ObjectEncoderOutputStream.writeObjectOverride(ObjectEncoderOutputStream.java:65)
> 	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
> 	at org.teiid.net.socket.OioOjbectChannelFactory$OioObjectChannel.write(OioOjbectChannelFactory.java:130)
> 	at org.teiid.net.socket.SocketServerInstanceImpl.send(SocketServerInstanceImpl.java:173)
> 	... 14 more
> But if we give less number of data in that param, e.g. 100x375, it works fine.
> Thanks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira