Configuring HttpTunnelingClientSocketChannel (Was: changes to HttpTunneling)

"이희승 (Trustin Lee)" trustin at gmail.com
Fri Aug 7 00:30:01 EDT 2009 

In your HttpTunnelingSCP.java, you added an ExceutionHandler with a
wrong Executor:

    pipeline.addLast("exec", new ExecutionHandler(threadPool));

threadPool above must be an OrderedMemoryAwareThreadPoolExecutor.
Otherwise the received data will be messed up, and that's why you were
getting such a exception.

Trustin

PS: Please post your question and bug report to netty-users at lists.jboss.org.

On 08/07/2009 12:30 AM, Jason Ward wrote:
> Hi Trustin - please find project attached.
> 
> I included the full stack trace in stacktrace.txt.
> 
> Notes :
>     - This project was build for use with Tomcat 6.0.x because it
> requires Comet processing. If you need to move to another servlet
> container, please feel free.
> 
>     - I found it very interesting that I use Netty elsewhere in the
> product to move large messages *not* using tunneling and everything
> works perfectly.  So this is very concerning that the Tunneling classes
> to not appear anywhere in the stack trace, but somehow seem related to
> the source of the problem.
> 
>     - I have verified with wireshark the XML on the wire is
> correct/intact. So I'm fairly certain it's actual software problems and
> not proxy/firewall issues. Altho, I can do more testing in this area if
> you disagree.
> 
> As always, thanks so much for your help!
> 
> -JW
> 
> Jason Ward wrote:
>> Hi Trustin - Reproduction might be difficult for you. I will see if I
>> can't work-up an eclipse project.
>>
>> The short answer is that we are pushing a very large XML msg (~11Mb)
>> thru this connection. Smaller msgs do not result in exceptions.
>>
>> Let me see if I can give you a small repro project. I will be in touch.
>>
>> Thanks,
>> JW
>>
>> 이희승 (Trustin Lee) wrote:
>>> Hi Jason,
>>>
>>> I didn't see such an issue so far.  Could you let me know how exactly I
>>> can reproduce the problem?  I can run the test by myself if necessary.
>>>
>>> Thanks in advance,
>>> Trustin
>>>
>>> On 08/05/2009 04:15 AM, Jason Ward wrote:
>>>   
>>>> FYI - this is the other exception I get regularly when trying to push
>>>> large XML msgs thru :
>>>>
>>>> [IoHandler] exceptionCaught()...
>>>> java.lang.ArrayIndexOutOfBoundsException
>>>>     at java.lang.System.arraycopy(Native Method)
>>>>     at
>>>> org.jboss.netty.buffer.HeapChannelBuffer.getBytes(HeapChannelBuffer.java:100)
>>>>     at
>>>> org.jboss.netty.buffer.DynamicChannelBuffer.getBytes(DynamicChannelBuffer.java:108)
>>>>     at
>>>> org.jboss.netty.buffer.HeapChannelBuffer.setBytes(HeapChannelBuffer.java:125)
>>>>     at
>>>> org.jboss.netty.buffer.DynamicChannelBuffer.setBytes(DynamicChannelBuffer.java:154)
>>>>     at
>>>> org.jboss.netty.buffer.AbstractChannelBuffer.discardReadBytes(AbstractChannelBuffer.java:119)
>>>>     at
>>>> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:219)
>>>>     at
>>>> org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:76)
>>>>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>> Source)
>>>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>     at java.lang.Thread.run(Unknown Source)
>>>> [IoHandler] exceptionCaught()...
>>>> java.lang.ArrayIndexOutOfBoundsException
>>>>     at java.lang.System.arraycopy(Native Method)
>>>>     at
>>>> org.jboss.netty.buffer.HeapChannelBuffer.getBytes(HeapChannelBuffer.java:100)
>>>>     at
>>>> org.jboss.netty.buffer.DynamicChannelBuffer.getBytes(DynamicChannelBuffer.java:108)
>>>>     at
>>>> org.jboss.netty.buffer.HeapChannelBuffer.setBytes(HeapChannelBuffer.java:125)
>>>>     at
>>>> org.jboss.netty.buffer.DynamicChannelBuffer.setBytes(DynamicChannelBuffer.java:154)
>>>>     at
>>>> org.jboss.netty.buffer.AbstractChannelBuffer.discardReadBytes(AbstractChannelBuffer.java:119)
>>>>     at
>>>> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:219)
>>>>     at
>>>> org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:76)
>>>>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>> Source)
>>>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>     at java.lang.Thread.run(Unknown Source)
>>>>
>>>> I'm not sure if there is a good way around this one or not. Do I need to
>>>> write my own ChannelBuff that handles the array copy in a better way?
>>>> (I'm not sure why the OOBException, but I will dig into it some more)
>>>>
>>>> Jason Ward wrote:
>>>>     
>>>>> Thanks Trustin - I made the changes and it works great. Sorry for
>>>>> missing this in the docs. I'll let you know if I see any issues with
>>>>> Tunneling and SSL.
>>>>>
>>>>> BTW - I'm finding some difficulties transferring large amounts of
>>>>> data. I get the following:
>>>>>
>>>>> [IoHandler] exceptionCaught()...
>>>>> org.jboss.netty.handler.codec.frame.TooLongFrameException: An HTTP
>>>>> line is larger than 4096 bytes.
>>>>>     at
>>>>> org.jboss.netty.handler.codec.http.HttpMessageDecoder.readLine(HttpMessageDecoder.java:518)
>>>>>     at
>>>>> org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:281)
>>>>>     at
>>>>> org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:85)
>>>>>     at
>>>>> org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:461)
>>>>>     at
>>>>> org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:434)
>>>>>     at
>>>>> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:342)
>>>>>     at
>>>>> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:329)
>>>>>     at
>>>>> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:330)
>>>>>     at
>>>>> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>>>>>     at
>>>>> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:203)
>>>>>     at
>>>>> org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:53)
>>>>>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>>> Source)
>>>>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>>     at java.lang.Thread.run(Unknown Source)
>>>>>
>>>>>
>>>>> I noticed from the thread "Sending large data without getting
>>>>> OutOfMemoryError" in the user group you've added
>>>>> ChunkedNioFile.java
>>>>> <http://n2.nabble.com/file/n3123729/ChunkedNioFile.java>
>>>>> and
>>>>> ChunkedNioStream.java
>>>>> <http://n2.nabble.com/file/n3123729/ChunkedNioStream.java>
>>>>> C <http://n2.nabble.com/file/n3123729/ChunkedNioStream.java>
>>>>> Are these classes I can take advantage of, or do you have a better
>>>>> suggestion for moving large amts of data thru my Tunnel?
>>>>>
>>>>> Thanks in advance,
>>>>> JW
>>>>>
>>>>> 이희승 (Trustin Lee) wrote:
>>>>>       
>>>>>> Hi Jason,
>>>>>>
>>>>>> You need to specify a couple more configuration options.  Check this
>>>>>> out: http://tinyurl.com/nu5b62
>>>>>>
>>>>>> HTH,
>>>>>> Trustin
>>>>>>
>>>>>>
>>>>>> On 07/29/2009 10:34 PM, Jason Ward wrote:
>>>>>>   
>>>>>>         
>>>>>>> Hi Trustin - Congrats on getting 3.1 to GA!
>>>>>>>
>>>>>>> Just a quick question. It seems you've revised the Tunneling support and
>>>>>>> HttpTunnelAddress is no longer included. Can you confirm this?
>>>>>>>
>>>>>>> I see now that in the example my code which works with CR2
>>>>>>>
>>>>>>>     channelFuture = bootstrap.connect(new HttpTunnelAddress(uri));
>>>>>>>
>>>>>>> needs to be adjusted
>>>>>>>
>>>>>>>     channelFuture = b.connect(new InetSocketAddress(uri.getHost(),
>>>>>>> uri.getPort()));
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Are there any other changes that I need to be aware of?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> JW
>>>>>>>     
>>>>>>>           
>>>>>>   
>>>>>>         
>>>
>>>

More information about the netty-users mailing list