Code example to write asynchronous server
manishr22
manishr22 at gmail.com
Mon Apr 20 03:34:44 EDT 2009
Hi yanky,
Please find the attached file of LoggingAgentNettyPipelineFactory.
Regards
Manish
LoggingAgentTCPNettyHandler.java
Hi:
Another class maybe u missed in zipped file:
LoggingAgentNettyPipelineFactory
yanky
2009/4/18 manishr22 <manishr22 at gmail.com>
>
> Hi,
>
> Trying to attach once again. otherwise i will copy post the java classes in
> the next post.
>
> Rest all description is given in previous post.
>
> Regards
> Manish netty+code.zip netty+code.zip
>
>
> Hi:
>
> why i can't see your attached code? maybe upload problem?
>
> yanky
>
> 2009/4/17 manishr22 <manishr22 at gmail.com>
>
> >
> >
> >
> > Hi:
> >
> >
> > Hi yanky,
> >
> > please find attached code-base for the netty server.
> >
> > As i mentioned before my client is normal TCP client and sending the
> > message in that is sending the data in following format
> >
> > [MESSAGE 1][4 byte message length][4 bytes header length][header
> data][body
> > data]
> >
> > [MESSAGE 2][4 byte message length][4 bytes header length][header
> data][body
> > data]
> >
> > [MESSAGE 3] [4 byte message length][4 bytes header length][header
> > data][body data]
> > ...
> > ...
> > ...
> >
> >
> >
> > Since my test client code is a bit clumsy.. like how the
> message-structure
> > has been created. so i am not attaching the test client code. But you can
> > test this from a normal TCp client by sending the data in the same way as
> i
> > mentioned above..
> >
> > then for more testing please try running multiple instances of client and
> > all are sending say 100 same-messages each and see how netty can handle
> > this.
> >
> >
> > Please let me know if you find problem in my server code and if you make
> > changes then send me back the code with the test clietn too. with whom
> you
> > will do the testing.
> >
> >
> > Your help is always appreciated..
> >
> > Thanks
> > Manish Ranjan netty+code.zip
> >
> >
> > Would you mind sending me your code? so that I can get to know what the
> > problem really is.
> >
> > thanks
> >
> > yanky
> >
> >
> > 2009/4/16 manishr22 <manishr22 at gmail.com>
> >
> > >
> > > Hi,
> > >
> > > Please suggest something about my problem.
> > >
> > > Regards
> > > Manish
> > >
> > > Hi yanky,
> > >
> > > The problem remains the same. I see my ChannelBuffer capacity is always
> > > 1024 though i am writing bigger data from client.So that again I see
> the
> > > same problem as before.
> > >
> > > Could you please help me in understanding the flow of decode() method?
> > > Though i read it but if you explain that may correct my understanding.
> > >
> > > As i understood, whenever buffer.resetReaderIndex(); encountered,
> > decode()
> > > method get called again, this means my message-length and header-length
> > > variable will again re-initialize with the first 4-4 bytes respectively
> > of
> > > the current buffer readIndex position. which should never happen. once
> i
> > red
> > > the header and message-length my application should first read that
> > message
> > > and then only go for next message reading operation.
> > >
> > > say my client is sending data like
> > >
> > > [MESSAGE 1][4 byte message length][4 bytes header length][header][body]
> > > [MESSAGE 2][4 byte message length][4 bytes header length][header][body]
> > > [MESSAGE 3] [4 byte message length][4 bytes header
> > > length][header][body].......
> > >
> > > Since multiple clients will se sending data in same fashion so my Netty
> > > server should understand them all.
> > >
> > > Note: Right now my test client is not written using netty api and
> simply
> > > writing message on socket, will that make any difference?
> > >
> > > I am really sorry to bug more, since i am learning Netty, so i have so
> > many
> > > questions.
> > >
> > > Regards
> > > Manish
> > >
> > >
> > >
> > >
> > > Hi:
> > >
> > > Maybe you can have a look at ChannelBuffer interface document. I am not
> > > sure
> > > I have got it right. But I am trying to correct your code according to
> my
> > > understanding. Some resetReaderIndex methods have been commented out
> for
> > > correction. You can try it. Please let me know if it works.
> > >
> > > good luck
> > >
> > > yanky
> > >
> > >
> > > public class LoggingAgentMessageDecoder extends FrameDecoder {
> > >
> > > private int maxMsgLenToRead =
> Constants.LOG_AGENT_MAXIMUM_MSG_LEN;
> > >
> > > private Logger logger = Logger.getLogger(this.getClass());
> > >
> > > @Override
> > > protected Object decode(ChannelHandlerContext ctx, Channel
> > channel,
> > > ChannelBuffer buffer) throws Exception {
> > >
> > > ChannelBuffer[] frame = new ChannelBuffer[2];
> > >
> > > // Make sure if the body length field was received.
> > > if (buffer.readableBytes() < 4) {
> > > return null;
> > > }
> > > // The length field is in the buffer.
> > >
> > > // Mark the current buffer position before reading the
> > length
> > > field
> > > buffer.markReaderIndex();
> > > // Read the length field.
> > > int messageLength = buffer.readInt();
> > > logger.info("Total message lenth is " + messageLength);
> > >
> > > // Make sure if the message length field was received.
> > > if (buffer.readableBytes() < 4) {
> > > // if not enough bytes, we should reset reader
> > index
> > > to ensure we can read header length field again at the next time when
> > this
> > > decode method is invoked
> > > buffer.resetReaderIndex();
> > > return null;
> > > }
> > > // buffer.markReaderIndex(); // not needed here
> > > int headerLength = buffer.readInt();
> > > logger.info("Total message lenth is " + headerLength);
> > >
> > > // buffer.resetReaderIndex(); // not needed here
> > > if (buffer.readableBytes() < headerLength) {
> > > // The whole bytes were not received yet - return
> > > null.
> > > // This method will be invoked again when more
> > > packets are
> > > // received and appended to the buffer.
> > >
> > > // Reset to the marked position to read the
> length
> > > field again
> > > // next time.
> > > buffer.resetReaderIndex(); // yes, this time we
> > need
> > > to reset reader index to read all bytes again
> > >
> > > return null;
> > > }
> > > frame[0] = buffer.readBytes(headerLength);
> > > // buffer.resetReaderIndex(); // not needed here
> > > System.out.println("Bufer capacity is " +
> > buffer.capacity());
> > > if (buffer.readableBytes() < messageLength) {
> > > // The whole bytes were not received yet - return
> > > null.
> > > // This method will be invoked again when more
> > > packets are
> > > // received and appended to the buffer.
> > >
> > > // Reset to the marked position to read the
> length
> > > field again
> > > // next time.
> > >
> > > // control is coming inside this since the
> capacity
> > > fo channelbuffer
> > > // is always less then the total message length.
> > > buffer.resetReaderIndex(); // yes, again we need
> to
> > > reset
> > >
> > > return null;
> > > }
> > >
> > > // There's enough bytes in the buffer. Read it.
> > > frame[1] = buffer.readBytes(messageLength);
> > > // TODO: need to add the logic to get the data parsed and
> > add
> > > into
> > > // activeAMQ
> > > return frame;
> > > }
> > > }
> > >
> > >
> > > 2009/4/14 manishr22 <manishr22 at gmail.com>
> > >
> > > >
> > > > Hi,
> > > >
> > > > Still this one is not able to solve this problem.
> > > >
> > > > The message comes from my server has proper format like:
> > > >
> > > > First 4 byte has information about body length. for example 1173
> > > > Next 4 byte has length of header for example 92
> > > > now in next 92 bytes i have my message header.
> > > > and then from 100th byte onward i have my message body till 1173
> byte.
> > > >
> > > > To solve this I have written my decoder and tried solving it like:
> > > >
> > > > public class LoggingAgentMessageDecoder extends FrameDecoder {
> > > >
> > > > private int maxMsgLenToRead =
> > Constants.LOG_AGENT_MAXIMUM_MSG_LEN;
> > > >
> > > > private Logger logger = Logger.getLogger(this.getClass());
> > > >
> > > > @Override
> > > > protected Object decode(ChannelHandlerContext ctx, Channel
> > > channel,
> > > > ChannelBuffer buffer) throws Exception {
> > > >
> > > > ChannelBuffer[] frame = new ChannelBuffer[2];
> > > >
> > > > // Make sure if the message length field was received.
> > > > if (buffer.readableBytes() < 4) {
> > > > return null;
> > > > }
> > > > // The length field is in the buffer.
> > > >
> > > > // Mark the current buffer position before reading the
> > > > length field
> > > > buffer.markReaderIndex();
> > > > // Read the length field.
> > > > int messageLength = buffer.readInt();
> > > > logger.info("Total message lenth is " +
> messageLength);
> > > >
> > > > // Make sure if the message length field was received.
> > > > if (buffer.readableBytes() < 4) {
> > > > return null;
> > > > }
> > > > buffer.markReaderIndex();
> > > > int headerLength = buffer.readInt();
> > > > logger.info("Total message lenth is " +
> headerLength);
> > > >
> > > > buffer.resetReaderIndex();
> > > > if (buffer.readableBytes() < headerLength) {
> > > > // The whole bytes were not received yet -
> > return
> > > > null.
> > > > // This method will be invoked again when more
> > > > packets are
> > > > // received and appended to the buffer.
> > > >
> > > > // Reset to the marked position to read the
> > length
> > > > field again
> > > > // next time.
> > > > buffer.resetReaderIndex();
> > > >
> > > > return null;
> > > > }
> > > > frame[0] = buffer.readBytes(headerLength);
> > > > buffer.resetReaderIndex();
> > > > System.out.println("Bufer capacity is " +
> > > > buffer.capacity());
> > > > if (buffer.readableBytes() < messageLength) {
> > > > // The whole bytes were not received yet -
> > return
> > > > null.
> > > > // This method will be invoked again when more
> > > > packets are
> > > > // received and appended to the buffer.
> > > >
> > > > // Reset to the marked position to read the
> > length
> > > > field again
> > > > // next time.
> > > >
> > > > // control is coming inside this since the
> > > capacity
> > > > fo channelbuffer
> > > > // is always less then the total message
> length.
> > > > buffer.resetReaderIndex();
> > > >
> > > > return null;
> > > > }
> > > >
> > > > // There's enough bytes in the buffer. Read it.
> > > > frame[1] = buffer.readBytes(messageLength);
> > > > // TODO: need to add the logic to get the data parsed
> > and
> > > > add into
> > > > // activeAMQ
> > > > return frame;
> > > > }
> > > > }
> > > >
> > > >
> > > > But the problem in this i see is whenever the controls come to if
> > > statement
> > > > which goes to read my body i see buffer.readableBytes() is always
> 1020
> > > and
> > > > since my body length is greater than this it goes inside the if
> > condition
> > > > and call the method buffer.resetReaderIndex().
> > > >
> > > > This actually recall the whole decode method again and then
> > re-initialize
> > > > the value for message-length and header-length to the wrong value
> like
> > > > message length as 92 and after that 262145.
> > > >
> > > > and this all mess up completely. for my testing from client i am
> > actually
> > > > sending only one data which message header of 92 byte and body as
> 1173
> > > > bytes.
> > > >
> > > > so i am not sure how come in different iteration the message length
> > gets
> > > > increased.
> > > >
> > > > i see always the capacity the the ChannelBuffer comes as 1024 though
> i
> > am
> > > > not setting the limit to channelBuffer.
> > > >
> > > >
> > > > Please suggest what wrong i am doing?
> > > >
> > > > Regards
> > > > Manish
> > > >
> > > >
> > > > Hi:
> > > >
> > > > You have a schema in your message. So while encoding and decoding,
> you
> > > have
> > > > to recognize a complete request or response message. A common
> practice
> > is
> > > > to
> > > > use another delimiter that's usually not used in protocol message to
> > mark
> > > > the boundary of different protocol messages. Fortunately, netty do
> has
> > > > out-of-box support for this problem. You can have a look at
> > > example.telnet
> > > > package and also codec.frame package to see how
> > > DelimiterBasedFrameDecoder
> > > > can work for you.
> > > >
> > > > Another consideration is that, you have a xml message, of course it
> is
> > > > string, but you can build your own encoder and decoder to translate
> > POJO
> > > > from/to xml string and then xml string from/to ChannelBuffer. That
> > would
> > > be
> > > > better. I think it would be like this:
> > > >
> > > > in your PipelineFactory:
> > > >
> > > > // this factory should be used for both client and server to setup
> > > pipeline
> > > > public ChannelPipeline getPipeline() throws Exception {
> > > > ChannelPipeline pipeline = pipeline();
> > > >
> > > > final int xmlMessageSizeLimit = 8192; // can be modified to
> > meet
> > > > your needs
> > > > // Add the text line codec combination first,
> > > > pipeline.addLast("framer", new DelimiterBasedFrameDecoder(
> > > > xmlMessageSizeLimit, Delimiters.nulDelimiter()));
> > > > pipeline.addLast("decoder", new MyProtocolDecoder());
> > > > pipeline.addLast("encoder", new MyProtocolEncoder());
> > > >
> > > > // and then business logic.
> > > > pipeline.addLast("handler", handler);
> > > > }
> > > >
> > > > and then define your own encoder and decoder:
> > > >
> > > > public class MyProtocolDecoder extends StringDecoder{
> > > > //just override decode method
> > > > protected Object decode(
> > > > ChannelHandlerContext ctx, Channel channel, Object msg)
> > throws
> > > > Exception {
> > > > String xml = super(ctx, channel, msg);
> > > > // then use any xml parser or xml binding tool to map xml to
> > your
> > > > POJO, I use XStream as example
> > > > XStream mapper = new XStream();
> > > > mapper.alias("header",
> > MyProtocolMessage.MyProtocolHeader.class);
> > > > mapper.alias("body", MyProtocolMessage.MyProtocolBody.class);
> > > > MyProtocolMessage protoMsg = mapper.fromXML(xml);
> > > > return protoMsg;
> > > > }
> > > > }
> > > >
> > > > public class MyProtocolMessage {
> > > > MyProtocolHeader header;
> > > > MyProtocolBody body;
> > > >
> > > > static class MyProtocolHeader {......}
> > > > static class MyProtocolBody {......}
> > > >
> > > > }
> > > >
> > > > you should do the reverse process in MyProtocolEncoder
> > > >
> > > > Hope it helps.
> > > >
> > > > good luck
> > > >
> > > > yanky
> > > >
> > > >
> > > > 2009/4/13 manishr22 <manishr22 at gmail.com>
> > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > How I can read a chunk of data from channel using netty api? my
> > client
> > > is
> > > > > writing every time on socket
> > > > >
> > > > > <manish> welcome to the real world</manish>
> > > > >
> > > > > and my netty server needs to read the message till </manish> in one
> > go.
> > > > and
> > > > > then next message starting from <manish> should be read by Netty
> > > server.
> > > > >
> > > > >
> > > > > Any code snippet would be a great help.
> > > > >
> > > > > In my decoder if i read the message from channelbuffer using
> > getbytes()
> > > > api
> > > > > i see the data but not able to get i can read the whole message in
> > one
> > > > go. i
> > > > > do see sometime some extra message being read from channel. any
> > reason
> > > > why
> > > > > it should be.
> > > > >
> > > > >
> > > > >
> > > > > Thanks in advance.
> > > > > Regards
> > > > > Manish
> > > > >
> > > > > Hi Yanky,
> > > > >
> > > > > Thanks so much for your guidance. I will look into factorial
> example
> > > will
> > > > > solving my problem based on this.
> > > > >
> > > > > For any further help, if needed i will post my request here.
> > > > >
> > > > > Thanks again
> > > > >
> > > > > Manish
> > > > >
> > > > >
> > > > >
> > > > > Thanks Yanky for the great answer! So nice that you are here as a
> > > > > part of the community.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > — Trustin Lee, http://gleamynode.net/
> > > > >
> > > > > On Sat, Apr 11, 2009 at 3:16 AM, yanky young <
> yanky.young at gmail.com>
> > > > > wrote:
> > > > > > Hi:
> > > > > >
> > > > > > I am not netty expert yet. I am just trying to give some hints.
> In
> > > your
> > > > > > case, you are actually building a request/response protocol. And
> > > > request
> > > > > and
> > > > > > response message may include quite complicated headers and body.
> So
> > > you
> > > > > have
> > > > > > to do some message encoding and decoding in both server side and
> > > client
> > > > > > side. You can see in netty codebase there are some out-of-box
> > encoder
> > > > and
> > > > > > decoder in handler.codec package. I suggest you take a look at
> > > > > > example.fractorial package to see how to implement your own
> encoder
> > > and
> > > > > > decoder for your message format which may be xml. If you are not
> > > > > sensitive
> > > > > > to message format, you can also have a look at example.localtime
> > > which
> > > > > use
> > > > > > google protobuf library as real encoder and decoder.
> > > > > >
> > > > > > good luck
> > > > > >
> > > > > > yanky
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2009/4/11 manishr22 <manishr22 at gmail.com>
> > > > > >>
> > > > > >> Hi Yanky,
> > > > > >>
> > > > > >> I took the sample code from svn trunk and saw the codes too.
> Could
> > > you
> > > > > >> please let me know which one among given sample code will fit
> for
> > > the
> > > > > >> reference of my requirement.
> > > > > >>
> > > > > >> Regards,
> > > > > >> Manish
> > > > > >>
> > > > > >> Hi:
> > > > > >>
> > > > > >> Of course there are some sample code in netty project. You can
> > just
> > > > > check
> > > > > >> out the svn trunk and see some samples in example package.
> > > > > >>
> > > > > >> good luck
> > > > > >>
> > > > > >> yanky
> > > > > >>
> > > > > >>
> > > > > >> 2009/4/10 manishr22 <manishr22 at gmail.com>
> > > > > >>
> > > > > >> >
> > > > > >> > Hi,
> > > > > >> >
> > > > > >> > I had a problem statement
> > > > > >> >
> > > > > >> > "to write an asynchronous TCP server to which multiple clients
> > > will
> > > > > >> > connect
> > > > > >> > and keep sending messages to my tcp server. My server should
> > read
> > > > the
> > > > > >> > message and send back some acknowledgment information to the
> > > client
> > > > > who
> > > > > >> > sent
> > > > > >> > the data. While reading the data at server since the message
> > comes
> > > > in
> > > > > >> > particular format i have to do some processing before sending
> > the
> > > > > >> > response
> > > > > >> > back to client."
> > > > > >> >
> > > > > >> > say my client is sending a message structure in which it has
> > some
> > > > > header
> > > > > >> > and then in body it has message like :
> > > > > >> >
> > > > > >> > <manish>welcome to the real world</manish>
> > > > > >> >
> > > > > >> > I was on the way to implement NIO but in between i found such
> a
> > > > great
> > > > > >> > library netty and thought of using it. i was trying to make
> use
> > of
> > > > > this
> > > > > >> > library with the given tutorial and examples but i found that
> > > > tutorial
> > > > > >> > is
> > > > > >> > not good enough to solve the complex problem.
> > > > > >> >
> > > > > >> > If you can provide me some guidance or sample code that how i
> > > should
> > > > > >> > solve
> > > > > >> > my problem using netty library, that would be great.
> > > > > >> >
> > > > > >> > Thanks in advance.
> > > > > >> >
> > > > > >> > Regards
> > > > > >> > Manish
> > > > > >> > --
> > > > > >> > View this message in context:
> > > > > >> >
> > > > > >> >
> > > > >
> > > >
> > >
> >
> http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2615889.html
> > > > > >> > Sent from the Netty User Group mailing list archive at
> > Nabble.com.
> > > > > >> >
> > > > > >> > _______________________________________________
> > > > > >> > netty-users mailing list
> > > > > >> > netty-users at lists.jboss.org
> > > > > >> > https://lists.jboss.org/mailman/listinfo/netty-users
> > > > > >> >
> > > > > >>
> > > > > >> _______________________________________________
> > > > > >> netty-users mailing list
> > > > > >> netty-users at lists.jboss.org
> > > > > >> https://lists.jboss.org/mailman/listinfo/netty-users
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> View this message in context:
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2617267.html
> > > > > >> Sent from the Netty User Group mailing list archive at
> Nabble.com.
> > > > > >>
> > > > > >> _______________________________________________
> > > > > >> netty-users mailing list
> > > > > >> netty-users at lists.jboss.org
> > > > > >> https://lists.jboss.org/mailman/listinfo/netty-users
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > netty-users mailing list
> > > > > > netty-users at lists.jboss.org
> > > > > > https://lists.jboss.org/mailman/listinfo/netty-users
> > > > > >
> > > > > >
> > > > >
> > > > > _______________________________________________
> > > > > netty-users mailing list
> > > > > netty-users at lists.jboss.org
> > > > > https://lists.jboss.org/mailman/listinfo/netty-users
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > View this message in context:
> > > > >
> > > >
> > >
> >
> http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2627351.html
> > > > > Sent from the Netty User Group mailing list archive at Nabble.com.
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > netty-users mailing list
> > > > > netty-users at lists.jboss.org
> > > > > https://lists.jboss.org/mailman/listinfo/netty-users
> > > > >
> > > >
> > > > _______________________________________________
> > > > netty-users mailing list
> > > > netty-users at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/netty-users
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > >
> > >
> >
> http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2632783.html
> > > > Sent from the Netty User Group mailing list archive at Nabble.com.
> > > >
> > > >
> > > > _______________________________________________
> > > > netty-users mailing list
> > > > netty-users at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/netty-users
> > > >
> > >
> > > _______________________________________________
> > > netty-users mailing list
> > > netty-users at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/netty-users
> > >
> > >
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> >
> http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2642120.html
> > > Sent from the Netty User Group mailing list archive at Nabble.com.
> > >
> > >
> > > _______________________________________________
> > > netty-users mailing list
> > > netty-users at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/netty-users
> > >
> >
> > _______________________________________________
> > netty-users mailing list
> > netty-users at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/netty-users
> >
> >
> >
> > --
> > View this message in context:
> >
> http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2650632.html
> > Sent from the Netty User Group mailing list archive at Nabble.com.
> >
> >
> > _______________________________________________
> > netty-users mailing list
> > netty-users at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/netty-users
> >
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>
>
>
> --
> View this message in context:
> http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2654456.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
>
>
> _______________________________________________
> netty-users mailing list
> netty-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>
_______________________________________________
netty-users mailing list
netty-users at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users
--
View this message in context: http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2662208.html
Sent from the Netty User Group mailing list archive at Nabble.com.
More information about the netty-users
mailing list