Structured message reading problem in Netty.
manishr22
manishr22 at gmail.com
Thu Apr 30 07:20:03 EDT 2009
Hi Trustin,
My decoder is :
//@ChannelPipelineCoverage("one") // tried with this option and by
commenting this
public class LoggingAgentMessageDecoder extends ReplayingDecoder<VoidEnum>/*
FrameDecoder */{
private int maxMsgLenToRead = 10000;
public static int flag = 0;
int messageLength = 0;
int headerLength = 0;
@Override
protected Object decode(ChannelHandlerContext ctx, Channel channel,
ChannelBuffer buffer, VoidEnum state) throws
Exception {
ChannelBuffer frame = null;
BufferData bData = new BufferData(); // this class has three
variable
// bodyLength : int
// headerLength : int
// buffer:
ChannelBuffer
// and their
getter-setter.
messageLength = buffer.readInt();
bData.setBodyLegth(messageLength);
headerLength = buffer.readInt();
bData.setHeaderLength(headerLength);
// return frame;
frame = buffer.readBytes(messageLength + headerLength);
bData.setBuffer(frame);
return bData;
// if (!readLength) {
// messageLength = buffer.readInt();
// headerLength = buffer.readInt();
// readLength = true;
// checkpoint();
// }
// if (readLength) {
// frame = buffer.readBytes(messageLength + headerLength);
// readLength = false;
// checkpoint();
//
// }
}
}
and my pipeline is:
@ChannelPipelineCoverage("all")
// tried will "one" option too
public class LoggingAgentNettyPipelineFactory extends SimpleChannelHandler {
private final ChannelHandler handler;
// private int port = 0;
public LoggingAgentNettyPipelineFactory(ChannelHandler handler) {
this.handler = handler;
}
// public LoggingAgentNettyPipelineFactory(final int port) {
// this.port = port;
// }
public ChannelPipeline getPipeline() throws Exception {
ChannelPipeline pipeline = pipeline();
// LoggingAgentTCPNettyHandler handler = new
// LoggingAgentTCPNettyHandler(
// this.port);
pipeline.addLast("decoder", new
LoggingAgentMessageDecoder());
pipeline.addLast("handler", this.handler);
return pipeline;
// Add the text line codec combination first,
// manish: 10000 is the maximum message size limit, this
should pick
// // from constant file
// ChannelPipeline pipeline = pipeline();
// pipeline.addLast("framer",
// new DelimiterBasedFrameDecoder(
// Constants.LOG_AGENT_MAXIMUM_MSG_LEN, Delimiters
// .lineDelimiter()));
// pipeline.addLast("decoder", new StringDecoder("UTF-8"));
// pipeline.addLast("encoder", new StringEncoder("UTF-8"));
//
// // and then business logic.
// pipeline.addLast("handler", handler);
//
// return pipeline;
}
}
In my handler message received method is like :
@Override
public void messageReceived(ChannelHandlerContext ctx,
MessageEvent messageEvent) {
int len;
int header;
BufferData buffData = null;
if (messageEvent.getMessage() instanceof BufferData) {
buffData = (BufferData) messageEvent.getMessage();
}
len = buffData.getBodyLegth();
header = buffData.getHeaderLength();
ChannelBuffer buff = buffData.getBuffer();
byte[] bytesRead = new byte[buff.capacity()];
buff.readBytes(bytesRead);
// Other business logic.
}
Hope i am doing everything correctly. Your advice will be highly
appreciated.
More observation, if i ran five client simultaneously and each client is
sending 10 messages, i see my handler receives messages one messages each
from all client and for the next message it gives problem and my handler
reports exception like:
30 Apr 09 12:44:36, INFO
com.dvclabs.logging.agent.LoggingAgentTCPNettyHandler: [New I/O server
worker #1-1] handleUpstream:120 HANDLE UPSTREAM [id: 0x01eb904d]
DISCONNECTED
30 Apr 09 12:44:36, INFO
com.dvclabs.logging.agent.LoggingAgentTCPNettyHandler: [New I/O server
worker #1-1] handleUpstream:120 HANDLE UPSTREAM [id: 0x01eb904d] UNBOUND
Though bootStrap i am providing like:
bootstrap.setOption("child.tcpNoDelay", true);
bootstrap.setOption("child.keepAlive", true);
May be this will help you understand my mistakes.
Regadrs,
Manish
Trustin Lee wrote:
>
> Hi Manish,
>
> Could you paste your decoder implementation and pipeline setup?
>
> Thanks in advance,
> Trustin
>
>
> manishr22 wrote:
>>
>> Hi Trustin,
>>
>> One small question:
>>
>> When I am doing the implementation the way you said like adding
>> ReplayDeocder, if i run whatever be the number of client with number of
>> messages from each, I see the first message from client read correctly
>> but from second message the header length and message lengh data read
>> incorrectly and then application crashes.
>>
>> Though i tested my application with the use of
>> "DelimiterBasedFrameDecoder" i see for all client i get correct data.
>> Since the string based delimiter in serialization and de-serialization
>> from string to byte and then int gives incorrect result.
>>
>> So i should be following the approach of ReplayingDecoder only but due to
>> above given problem i am facing issue.
>> In replayingDecoder i tried with the method checkPoint() too. I think
>> everytime when data is read from channelBuffer the readerIndex() value
>> increase. so checkpoint() here shouldn't be necessary. Though i am
>> compromising with the performance for sometime so that same data get read
>> again and again for the time being.
>>
>> IF first message read correctly it should happen for rest messages too.
>>
>> i have tried for @ChannelPipelineCoverage("all") and
>> @ChannelPipelineCoverage("one") both. but the problem is same.
>>
>> Any suggestion?
>>
>> Regards
>> Manish Ranjan
>>
>> manishr22 wrote:
>>>
>>> Hi Trustin,
>>>
>>> Thanks for your guidance. This should help me out to solve my problem.
>>> You people of created a great library, just more of documentation and
>>> example will help any developer a lot to use Netty.
>>>
>>> Regards
>>> Manish
>>>
>>>
>>> Trustin Lee-2 wrote:
>>>>
>>>> Hi Manish,
>>>>
>>>> I hope you are not discouraged by previous comment.
>>>>
>>>> I'd like to recommend you to use ReplayingDecoder in this case.
>>>>
>>>> public class MyDecoder extends ReplayingDecode<VoidEnum> {
>>>> @Override
>>>> protected Object decode(..., buf, ...) {
>>>> int messageLength = buf.readInt();
>>>> int headerLength = buf.readInt();
>>>> return buf.readBytes(messageLength + headerLength);
>>>> }
>>>> }
>>>>
>>>> Than the handler placed after 'MyDecoder' will get a ChannelBuffer
>>>> with expected length. For example, if a message whose messageLength
>>>> is 10 and headerLength is 6, you will get a ChannelBuffer whose
>>>> readable bytes are 4 + 4 + 6 + 10.
>>>>
>>>> Let me know if you have more questions. :)
>>>>
>>>> HTH,
>>>>
>>>> — Trustin Lee, http://gleamynode.net/
>>>>
>>>> On Fri, Apr 24, 2009 at 2:16 PM, manishr22 <manishr22 at gmail.com> wrote:
>>>>>
>>>>> Hi Trustin,
>>>>>
>>>>> Sorry to bug you guys much.
>>>>>
>>>>> I did read the explanation about FrameDecoder and did the earlier
>>>>> implementation using the same. In my earlier post i gave those source
>>>>> code
>>>>> too. Since that was also having problem in buffer's capacity() and
>>>>> resetReaderIndex().
>>>>>
>>>>> I am sorry to give an impression like i wanted to get my code fixed by
>>>>> someone else, but i never had this intention, i only wanted to get the
>>>>> advice and opinion from Netty expert, since i was learning this.
>>>>>
>>>>> Regards
>>>>> Manish
>>>>>
>>>>> Trustin Lee-2 wrote:
>>>>>>
>>>>>> Hi Manish,
>>>>>>
>>>>>> Looking from your code, you don't have a FrameDecoder or
>>>>>> ReplayingDecoder in a pipeline, which means you have a good chance to
>>>>>> receive only partial data, which leads to IndexOutOfBoundsException.
>>>>>> Did you have a chance to read the first chapter of the user guide?
>>>>>> There's good amount of explanation about this issue there.
>>>>>>
>>>>>> You and other community members have exchanged quite a lot of
>>>>>> questions and answers so far. My impression so far is that you just
>>>>>> want people to fix your code. We could solve your problem just by
>>>>>> writing the fixed code for you, but it would be much better for you
>>>>>> to
>>>>>> follow all the step demonstrated in the user guide. Then you will
>>>>>> know what is wrong with your current code very easily.
>>>>>>
>>>>>> Also, you might find the factorial example more useful than the
>>>>>> discard example for your case.
>>>>>>
>>>>>> — Trustin Lee, http://gleamynode.net/
>>>>>>
>>>>>> On Thu, Apr 23, 2009 at 8:45 PM, manishr22 <manishr22 at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am new to netty and tried writing netty-server as per my
>>>>>>> requirement
>>>>>>> seeing various examples. But i am not getting the result correctly.
>>>>>>>
>>>>>>>
>>>>>>> I am trying to write a netty server that can read the that will read
>>>>>>> the
>>>>>>> message-structre from normal synchronous multi-threaded client ( say
>>>>>>> 10
>>>>>>> client is sending 1000 messages each ) to the server.
>>>>>>>
>>>>>>>
>>>>>>> By message structure i mean my client is sending the message in the
>>>>>>> format :
>>>>>>>
>>>>>>>
>>>>>>> [MESSAGE 1-CLIENT 1] message-length(first 4 byte) header
>>>>>>> length(next 4
>>>>>>> byte) data (including
>>>>>>>
>>>>>>> header and body as specified bytes of message length and header
>>>>>>> length)
>>>>>>>
>>>>>>>
>>>>>>> [MESSAGE 2-CLIENT 1] message-length(first 4 byte) header
>>>>>>> length(next 4
>>>>>>> byte) data (including
>>>>>>>
>>>>>>> header and body as specified bytes of message length and header
>>>>>>> length)
>>>>>>>
>>>>>>>
>>>>>>> ........
>>>>>>>
>>>>>>> ........
>>>>>>>
>>>>>>> [MESSAGE 1-CLIENT 2] message-length(first 4 byte) header
>>>>>>> length(next 4
>>>>>>> byte) data (including
>>>>>>>
>>>>>>> header and body as specified bytes of message length and header
>>>>>>> length)
>>>>>>>
>>>>>>>
>>>>>>> [MESSAGE 2-CLIENT 2] message-length(first 4 byte) header
>>>>>>> length(next 4
>>>>>>> byte) data (including
>>>>>>>
>>>>>>> header and body as specified bytes of message length and header
>>>>>>> length)
>>>>>>>
>>>>>>> ....
>>>>>>> .....
>>>>>>>
>>>>>>> [For N number of client and N
>>>>>>> http://n2.nabble.com/file/n2682454/netty.zip
>>>>>>> netty.zip number of messages]
>>>>>>>
>>>>>>> and now my netty server should read all the different messages from
>>>>>>> client/s
>>>>>>> and do the proper message parsing and do some business operation.
>>>>>>>
>>>>>>>
>>>>>>> How i can implement the reading of message-structure using netty.
>>>>>>>
>>>>>>>
>>>>>>> I saw several example and implemented something in it. but its not
>>>>>>> giving
>>>>>>> the appropriate result rather i am getting exception all the time.
>>>>>>>
>>>>>>> The basic source code which i tried for this is also attached for
>>>>>>> reference.
>>>>>>>
>>>>>>> Any suggestion and help with some sample code will be really
>>>>>>> appreciated.
>>>>>>>
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>> Manish
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://n2.nabble.com/Structured-message-reading-problem-in-Netty.-tp2682454p2682454.html
>>>>>>> Sent from the Netty User Group mailing list archive at Nabble.com.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> netty-users mailing list
>>>>>>> netty-users at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> netty-users mailing list
>>>>>> netty-users at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://n2.nabble.com/Structured-message-reading-problem-in-Netty.-tp2682454p2689756.html
>>>>> Sent from the Netty User Group mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> netty-users mailing list
>>>>> netty-users at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>>
>>>>
>>>> _______________________________________________
>>>> netty-users mailing list
>>>> netty-users at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>
>>>>
>>>
>>>
>>
>>
>
>
--
View this message in context: http://n2.nabble.com/Structured-message-reading-problem-in-Netty.-tp2682454p2746459.html
Sent from the Netty User Group mailing list archive at Nabble.com.
More information about the netty-users
mailing list