Structured message reading problem in Netty.
manishr22
manishr22 at gmail.com
Fri May 1 00:26:20 EDT 2009
Hi Hi Trustin,
I am not able to post anything on forum. it remains in pending state so
tried sending email response.
My decoder is :
//@ChannelPipelineCoverage("one") // tried with this option and by
commenting this
public class LoggingAgentMessageDecoder extends ReplayingDecoder<VoidEnum>/*
FrameDecoder */{
private int maxMsgLenToRead = 10000;
public static int flag = 0;
int messageLength = 0;
int headerLength = 0;
@Override
protected Object decode(ChannelHandlerContext ctx, Channel channel,
ChannelBuffer buffer, VoidEnum state) throws
Exception {
ChannelBuffer frame = null;
BufferData bData = new BufferData(); // this class has three
variable
// bodyLength : int
// headerLength : int
// buffer:
ChannelBuffer
// and their
getter-setter.
messageLength = buffer.readInt();
bData.setBodyLegth(messageLength);
headerLength = buffer.readInt();
bData.setHeaderLength(headerLength);
// return frame;
frame = buffer.readBytes(messageLength + headerLength);
bData.setBuffer(frame);
return bData;
// if (!readLength) {
// messageLength = buffer.readInt();
// headerLength = buffer.readInt();
// readLength = true;
// checkpoint();
// }
// if (readLength) {
// frame = buffer.readBytes(messageLength + headerLength);
// readLength = false;
// checkpoint();
//
// }
}
}
and my pipeline is:
@ChannelPipelineCoverage("all")
// tried will "one" option too
public class LoggingAgentNettyPipelineFactory extends SimpleChannelHandler {
private final ChannelHandler handler;
// private int port = 0;
public LoggingAgentNettyPipelineFactory(ChannelHandler handler) {
this.handler = handler;
}
// public LoggingAgentNettyPipelineFactory(final int port) {
// this.port = port;
// }
public ChannelPipeline getPipeline() throws Exception {
ChannelPipeline pipeline = pipeline();
// LoggingAgentTCPNettyHandler handler = new
// LoggingAgentTCPNettyHandler(
// this.port);
pipeline.addLast("decoder", new
LoggingAgentMessageDecoder());
pipeline.addLast("handler", this.handler);
return pipeline;
// Add the text line codec combination first,
// manish: 10000 is the maximum message size limit, this
should pick
// // from constant file
// ChannelPipeline pipeline = pipeline();
// pipeline.addLast("framer",
// new DelimiterBasedFrameDecoder(
// Constants.LOG_AGENT_MAXIMUM_MSG_LEN, Delimiters
// .lineDelimiter()));
// pipeline.addLast("decoder", new StringDecoder("UTF-8"));
// pipeline.addLast("encoder", new StringEncoder("UTF-8"));
//
// // and then business logic.
// pipeline.addLast("handler", handler);
//
// return pipeline;
}
}
In my handler message received method is like :
@Override
public void messageReceived(ChannelHandlerContext ctx,
MessageEvent messageEvent) {
int len;
int header;
BufferData buffData = null;
if (messageEvent.getMessage() instanceof BufferData) {
buffData = (BufferData) messageEvent.getMessage();
}
len = buffData.getBodyLegth();
header = buffData.getHeaderLength();
ChannelBuffer buff = buffData.getBuffer();
byte[] bytesRead = new byte[buff.capacity()];
buff.readBytes(bytesRead);
// Other business logic.
}
Hope i am doing everything correctly. Your advice will be highly
appreciated.
More observation, if i ran five client simultaneously and each client is
sending 10 messages, i see my handler receives messages one messages each
from all client and for the next message it gives problem and my handler
reports exception like:
30 Apr 09 12:44:36, INFO
com.dvclabs.logging.agent.LoggingAgentTCPNettyHandler: [New I/O server
worker #1-1] handleUpstream:120 HANDLE UPSTREAM [id: 0x01eb904d]
DISCONNECTED
30 Apr 09 12:44:36, INFO
com.dvclabs.logging.agent.LoggingAgentTCPNettyHandler: [New I/O server
worker #1-1] handleUpstream:120 HANDLE UPSTREAM [id: 0x01eb904d] UNBOUND
Though bootStrap i am providing like:
bootstrap.setOption("child.tcpNoDelay", true);
bootstrap.setOption("child.keepAlive", true);
May be this will help you understand my mistakes.
Regadrs,
Manish
On Wed, Apr 29, 2009 at 9:02 PM, Trustin Lee (via Nabble) <
ml-user+57722-790643042 at n2.nabble.com<ml-user%2B57722-790643042 at n2.nabble.com>
> wrote:
> Hi Manish,
>
> Could you paste your decoder implementation and pipeline setup?
>
> Thanks in advance,
> Trustin
>
> manishr22 wrote:
> Hi Trustin,
>
> One small question:
>
> When I am doing the implementation the way you said like adding
> ReplayDeocder, if i run whatever be the number of client with number of
> messages from each, I see the first message from client read correctly but
> from second message the header length and message lengh data read
> incorrectly and then application crashes.
>
> Though i tested my application with the use of "DelimiterBasedFrameDecoder"
> i see for all client i get correct data. Since the string based delimiter in
> serialization and de-serialization from string to byte and then int gives
> incorrect result.
>
> So i should be following the approach of ReplayingDecoder only but due to
> above given problem i am facing issue.
> In replayingDecoder i tried with the method checkPoint() too. I think
> everytime when data is read from channelBuffer the readerIndex() value
> increase. so checkpoint() here shouldn't be necessary. Though i am
> compromising with the performance for sometime so that same data get read
> again and again for the time being.
>
> IF first message read correctly it should happen for rest messages too.
>
> i have tried for @ChannelPipelineCoverage("all") and
> @ChannelPipelineCoverage("one") both. but the problem is same.
>
> Any suggestion?
>
> Regards
> Manish Ranjan
>
> manishr22 wrote:
> Hi Trustin,
>
> Thanks for your guidance. This should help me out to solve my problem. You
> people of created a great library, just more of documentation and example
> will help any developer a lot to use Netty.
>
> Regards
> Manish
>
> Trustin Lee-2 wrote:
> Hi Manish,
>
> I hope you are not discouraged by previous comment.
>
> I'd like to recommend you to use ReplayingDecoder in this case.
>
> public class MyDecoder extends ReplayingDecode<VoidEnum> {
> @Override
> protected Object decode(..., buf, ...) {
> int messageLength = buf.readInt();
> int headerLength = buf.readInt();
> return buf.readBytes(messageLength + headerLength);
> }
> }
>
> Than the handler placed after 'MyDecoder' will get a ChannelBuffer
> with expected length. For example, if a message whose messageLength
> is 10 and headerLength is 6, you will get a ChannelBuffer whose
> readable bytes are 4 + 4 + 6 + 10.
>
> Let me know if you have more questions. :)
>
> HTH,
>
> — Trustin Lee, http://gleamynode.net/
>
> On Fri, Apr 24, 2009 at 2:16 PM, manishr22 <manishr22 at gmail.com> wrote:
> >
> > Hi Trustin,
> >
> > Sorry to bug you guys much.
> >
> > I did read the explanation about FrameDecoder and did the earlier
> > implementation using the same. In my earlier post i gave those source
> code
> > too. Since that was also having problem in buffer's capacity() and
> > resetReaderIndex().
> >
> > I am sorry to give an impression like i wanted to get my code fixed by
> > someone else, but i never had this intention, i only wanted to get the
> > advice and opinion from Netty expert, since i was learning this.
> >
> > Regards
> > Manish
> >
> > Trustin Lee-2 wrote:
> >>
> >> Hi Manish,
> >>
> >> Looking from your code, you don't have a FrameDecoder or
> >> ReplayingDecoder in a pipeline, which means you have a good chance to
> >> receive only partial data, which leads to IndexOutOfBoundsException.
> >> Did you have a chance to read the first chapter of the user guide?
> >> There's good amount of explanation about this issue there.
> >>
> >> You and other community members have exchanged quite a lot of
> >> questions and answers so far. My impression so far is that you just
> >> want people to fix your code. We could solve your problem just by
> >> writing the fixed code for you, but it would be much better for you to
> >> follow all the step demonstrated in the user guide. Then you will
> >> know what is wrong with your current code very easily.
> >>
> >> Also, you might find the factorial example more useful than the
> >> discard example for your case.
> >>
> >> — Trustin Lee, http://gleamynode.net/
> >>
> >> On Thu, Apr 23, 2009 at 8:45 PM, manishr22 <manishr22 at gmail.com> wrote:
>
> >>>
> >>> Hi,
> >>>
> >>> I am new to netty and tried writing netty-server as per my requirement
> >>> seeing various examples. But i am not getting the result correctly.
> >>>
> >>>
> >>> I am trying to write a netty server that can read the that will read
> the
> >>> message-structre from normal synchronous multi-threaded client ( say 10
>
> >>> client is sending 1000 messages each ) to the server.
> >>>
> >>>
> >>> By message structure i mean my client is sending the message in the
> >>> format :
> >>>
> >>>
> >>> [MESSAGE 1-CLIENT 1] message-length(first 4 byte) header length(next
> 4
> >>> byte) data (including
> >>>
> >>> header and body as specified bytes of message length and header length)
>
> >>>
> >>>
> >>> [MESSAGE 2-CLIENT 1] message-length(first 4 byte) header length(next
> 4
> >>> byte) data (including
> >>>
> >>> header and body as specified bytes of message length and header length)
>
> >>>
> >>>
> >>> ........
> >>>
> >>> ........
> >>>
> >>> [MESSAGE 1-CLIENT 2] message-length(first 4 byte) header length(next
> 4
> >>> byte) data (including
> >>>
> >>> header and body as specified bytes of message length and header length)
>
> >>>
> >>>
> >>> [MESSAGE 2-CLIENT 2] message-length(first 4 byte) header length(next
> 4
> >>> byte) data (including
> >>>
> >>> header and body as specified bytes of message length and header length)
>
> >>>
> >>> ....
> >>> .....
> >>>
> >>> [For N number of client and N
> >>> http://n2.nabble.com/file/n2682454/netty.zip
> >>> netty.zip number of messages]
> >>>
> >>> and now my netty server should read all the different messages from
> >>> client/s
> >>> and do the proper message parsing and do some business operation.
> >>>
> >>>
> >>> How i can implement the reading of message-structure using netty.
> >>>
> >>>
> >>> I saw several example and implemented something in it. but its not
> giving
> >>> the appropriate result rather i am getting exception all the time.
> >>>
> >>> The basic source code which i tried for this is also attached for
> >>> reference.
> >>>
> >>> Any suggestion and help with some sample code will be really
> appreciated.
> >>>
> >>>
> >>> Regards
> >>>
> >>> Manish
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>>
> http://n2.nabble.com/Structured-message-reading-problem-in-Netty.-tp2682454p2682454.html
> >>> Sent from the Netty User Group mailing list archive at Nabble.com.
> >>>
> >>> _______________________________________________
> >>> netty-users mailing list
> >>> netty-users at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/netty-users
> >>>
> >>
> >> _______________________________________________
> >> netty-users mailing list
> >> netty-users at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/netty-users
> >>
> >>
> >
> > --
> > View this message in context:
> http://n2.nabble.com/Structured-message-reading-problem-in-Netty.-tp2682454p2689756.html
> > Sent from the Netty User Group mailing list archive at Nabble.com.
> >
> >
> > _______________________________________________
> > netty-users mailing list
> > netty-users at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/netty-users
> >
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>
> Trustin Lee, Principal Software Engineer, JBoss, a division of Red Hat
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
>
>
> ------------------------------
> This email is a reply to your post @
> http://n2.nabble.com/Structured-message-reading-problem-in-Netty.-tp2682454p2741285.html
> You can reply by email or by visting the link above.
>
>
--
View this message in context: http://n2.nabble.com/Structured-message-reading-problem-in-Netty.-tp2682454p2751687.html
Sent from the Netty User Group mailing list archive at Nabble.com.
More information about the netty-users
mailing list