Hi:<br><br>Would you mind sending me your code? so that I can get to know what the problem really is.<br><br>thanks<br><br>yanky<br><br><br><div class="gmail_quote">2009/4/16 manishr22 <span dir="ltr"><<a href="mailto:manishr22@gmail.com">manishr22@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
Hi,<br>
<br>
Please suggest something about my problem.<br>
<br>
Regards<br>
Manish<br>
<div><div></div><div class="h5"><br>
Hi yanky,<br>
<br>
The problem remains the same. I see my ChannelBuffer capacity is always 1024 though i am writing bigger data from client.So that again I see the same problem as before.<br>
<br>
Could you please help me in understanding the flow of decode() method? Though i read it but if you explain that may correct my understanding.<br>
<br>
As i understood, whenever buffer.resetReaderIndex(); encountered, decode() method get called again, this means my message-length and header-length variable will again re-initialize with the first 4-4 bytes respectively of the current buffer readIndex position. which should never happen. once i red the header and message-length my application should first read that message and then only go for next message reading operation.<br>
<br>
say my client is sending data like<br>
<br>
[MESSAGE 1][4 byte message length][4 bytes header length][header][body] [MESSAGE 2][4 byte message length][4 bytes header length][header][body] [MESSAGE 3] [4 byte message length][4 bytes header length][header][body].......<br>
<br>
Since multiple clients will se sending data in same fashion so my Netty server should understand them all.<br>
<br>
Note: Right now my test client is not written using netty api and simply writing message on socket, will that make any difference?<br>
<br>
I am really sorry to bug more, since i am learning Netty, so i have so many questions.<br>
<br>
Regards<br>
Manish<br>
<br>
<br>
<br>
<br>
Hi:<br>
<br>
Maybe you can have a look at ChannelBuffer interface document. I am not sure<br>
I have got it right. But I am trying to correct your code according to my<br>
understanding. Some resetReaderIndex methods have been commented out for<br>
correction. You can try it. Please let me know if it works.<br>
<br>
good luck<br>
<br>
yanky<br>
<br>
<br>
public class LoggingAgentMessageDecoder extends FrameDecoder {<br>
<br>
private int maxMsgLenToRead = Constants.LOG_AGENT_MAXIMUM_MSG_LEN;<br>
<br>
private Logger logger = Logger.getLogger(this.getClass());<br>
<br>
@Override<br>
protected Object decode(ChannelHandlerContext ctx, Channel channel,<br>
ChannelBuffer buffer) throws Exception {<br>
<br>
ChannelBuffer[] frame = new ChannelBuffer[2];<br>
<br>
// Make sure if the body length field was received.<br>
if (buffer.readableBytes() < 4) {<br>
return null;<br>
}<br>
// The length field is in the buffer.<br>
<br>
// Mark the current buffer position before reading the length<br>
field<br>
buffer.markReaderIndex();<br>
// Read the length field.<br>
int messageLength = buffer.readInt();<br>
<a href="http://logger.info" target="_blank">logger.info</a>("Total message lenth is " + messageLength);<br>
<br>
// Make sure if the message length field was received.<br>
if (buffer.readableBytes() < 4) {<br>
// if not enough bytes, we should reset reader index<br>
to ensure we can read header length field again at the next time when this<br>
decode method is invoked<br>
buffer.resetReaderIndex();<br>
return null;<br>
}<br>
// buffer.markReaderIndex(); // not needed here<br>
int headerLength = buffer.readInt();<br>
<a href="http://logger.info" target="_blank">logger.info</a>("Total message lenth is " + headerLength);<br>
<br>
// buffer.resetReaderIndex(); // not needed here<br>
if (buffer.readableBytes() < headerLength) {<br>
// The whole bytes were not received yet - return<br>
null.<br>
// This method will be invoked again when more<br>
packets are<br>
// received and appended to the buffer.<br>
<br>
// Reset to the marked position to read the length<br>
field again<br>
// next time.<br>
buffer.resetReaderIndex(); // yes, this time we need<br>
to reset reader index to read all bytes again<br>
<br>
return null;<br>
}<br>
frame[0] = buffer.readBytes(headerLength);<br>
// buffer.resetReaderIndex(); // not needed here<br>
System.out.println("Bufer capacity is " + buffer.capacity());<br>
if (buffer.readableBytes() < messageLength) {<br>
// The whole bytes were not received yet - return<br>
null.<br>
// This method will be invoked again when more<br>
packets are<br>
// received and appended to the buffer.<br>
<br>
// Reset to the marked position to read the length<br>
field again<br>
// next time.<br>
<br>
// control is coming inside this since the capacity<br>
fo channelbuffer<br>
// is always less then the total message length.<br>
buffer.resetReaderIndex(); // yes, again we need to<br>
reset<br>
<br>
return null;<br>
}<br>
<br>
// There's enough bytes in the buffer. Read it.<br>
frame[1] = buffer.readBytes(messageLength);<br>
// TODO: need to add the logic to get the data parsed and add<br>
into<br>
// activeAMQ<br>
return frame;<br>
}<br>
}<br>
<br>
<br>
2009/4/14 manishr22 <<a href="mailto:manishr22@gmail.com">manishr22@gmail.com</a>><br>
<br>
><br>
> Hi,<br>
><br>
> Still this one is not able to solve this problem.<br>
><br>
> The message comes from my server has proper format like:<br>
><br>
> First 4 byte has information about body length. for example 1173<br>
> Next 4 byte has length of header for example 92<br>
> now in next 92 bytes i have my message header.<br>
> and then from 100th byte onward i have my message body till 1173 byte.<br>
><br>
> To solve this I have written my decoder and tried solving it like:<br>
><br>
> public class LoggingAgentMessageDecoder extends FrameDecoder {<br>
><br>
> private int maxMsgLenToRead = Constants.LOG_AGENT_MAXIMUM_MSG_LEN;<br>
><br>
> private Logger logger = Logger.getLogger(this.getClass());<br>
><br>
> @Override<br>
> protected Object decode(ChannelHandlerContext ctx, Channel channel,<br>
> ChannelBuffer buffer) throws Exception {<br>
><br>
> ChannelBuffer[] frame = new ChannelBuffer[2];<br>
><br>
> // Make sure if the message length field was received.<br>
> if (buffer.readableBytes() < 4) {<br>
> return null;<br>
> }<br>
> // The length field is in the buffer.<br>
><br>
> // Mark the current buffer position before reading the<br>
> length field<br>
> buffer.markReaderIndex();<br>
> // Read the length field.<br>
> int messageLength = buffer.readInt();<br>
> <a href="http://logger.info" target="_blank">logger.info</a>("Total message lenth is " + messageLength);<br>
><br>
> // Make sure if the message length field was received.<br>
> if (buffer.readableBytes() < 4) {<br>
> return null;<br>
> }<br>
> buffer.markReaderIndex();<br>
> int headerLength = buffer.readInt();<br>
> <a href="http://logger.info" target="_blank">logger.info</a>("Total message lenth is " + headerLength);<br>
><br>
> buffer.resetReaderIndex();<br>
> if (buffer.readableBytes() < headerLength) {<br>
> // The whole bytes were not received yet - return<br>
> null.<br>
> // This method will be invoked again when more<br>
> packets are<br>
> // received and appended to the buffer.<br>
><br>
> // Reset to the marked position to read the length<br>
> field again<br>
> // next time.<br>
> buffer.resetReaderIndex();<br>
><br>
> return null;<br>
> }<br>
> frame[0] = buffer.readBytes(headerLength);<br>
> buffer.resetReaderIndex();<br>
> System.out.println("Bufer capacity is " +<br>
> buffer.capacity());<br>
> if (buffer.readableBytes() < messageLength) {<br>
> // The whole bytes were not received yet - return<br>
> null.<br>
> // This method will be invoked again when more<br>
> packets are<br>
> // received and appended to the buffer.<br>
><br>
> // Reset to the marked position to read the length<br>
> field again<br>
> // next time.<br>
><br>
> // control is coming inside this since the capacity<br>
> fo channelbuffer<br>
> // is always less then the total message length.<br>
> buffer.resetReaderIndex();<br>
><br>
> return null;<br>
> }<br>
><br>
> // There's enough bytes in the buffer. Read it.<br>
> frame[1] = buffer.readBytes(messageLength);<br>
> // TODO: need to add the logic to get the data parsed and<br>
> add into<br>
> // activeAMQ<br>
> return frame;<br>
> }<br>
> }<br>
><br>
><br>
> But the problem in this i see is whenever the controls come to if statement<br>
> which goes to read my body i see buffer.readableBytes() is always 1020 and<br>
> since my body length is greater than this it goes inside the if condition<br>
> and call the method buffer.resetReaderIndex().<br>
><br>
> This actually recall the whole decode method again and then re-initialize<br>
> the value for message-length and header-length to the wrong value like<br>
> message length as 92 and after that 262145.<br>
><br>
> and this all mess up completely. for my testing from client i am actually<br>
> sending only one data which message header of 92 byte and body as 1173<br>
> bytes.<br>
><br>
> so i am not sure how come in different iteration the message length gets<br>
> increased.<br>
><br>
> i see always the capacity the the ChannelBuffer comes as 1024 though i am<br>
> not setting the limit to channelBuffer.<br>
><br>
><br>
> Please suggest what wrong i am doing?<br>
><br>
> Regards<br>
> Manish<br>
><br>
><br>
> Hi:<br>
><br>
> You have a schema in your message. So while encoding and decoding, you have<br>
> to recognize a complete request or response message. A common practice is<br>
> to<br>
> use another delimiter that's usually not used in protocol message to mark<br>
> the boundary of different protocol messages. Fortunately, netty do has<br>
> out-of-box support for this problem. You can have a look at example.telnet<br>
> package and also codec.frame package to see how DelimiterBasedFrameDecoder<br>
> can work for you.<br>
><br>
> Another consideration is that, you have a xml message, of course it is<br>
> string, but you can build your own encoder and decoder to translate POJO<br>
> from/to xml string and then xml string from/to ChannelBuffer. That would be<br>
> better. I think it would be like this:<br>
><br>
> in your PipelineFactory:<br>
><br>
> // this factory should be used for both client and server to setup pipeline<br>
> public ChannelPipeline getPipeline() throws Exception {<br>
> ChannelPipeline pipeline = pipeline();<br>
><br>
> final int xmlMessageSizeLimit = 8192; // can be modified to meet<br>
> your needs<br>
> // Add the text line codec combination first,<br>
> pipeline.addLast("framer", new DelimiterBasedFrameDecoder(<br>
> xmlMessageSizeLimit, Delimiters.nulDelimiter()));<br>
> pipeline.addLast("decoder", new MyProtocolDecoder());<br>
> pipeline.addLast("encoder", new MyProtocolEncoder());<br>
><br>
> // and then business logic.<br>
> pipeline.addLast("handler", handler);<br>
> }<br>
><br>
> and then define your own encoder and decoder:<br>
><br>
> public class MyProtocolDecoder extends StringDecoder{<br>
> //just override decode method<br>
> protected Object decode(<br>
> ChannelHandlerContext ctx, Channel channel, Object msg) throws<br>
> Exception {<br>
> String xml = super(ctx, channel, msg);<br>
> // then use any xml parser or xml binding tool to map xml to your<br>
> POJO, I use XStream as example<br>
> XStream mapper = new XStream();<br>
> mapper.alias("header", MyProtocolMessage.MyProtocolHeader.class);<br>
> mapper.alias("body", MyProtocolMessage.MyProtocolBody.class);<br>
> MyProtocolMessage protoMsg = mapper.fromXML(xml);<br>
> return protoMsg;<br>
> }<br>
> }<br>
><br>
> public class MyProtocolMessage {<br>
> MyProtocolHeader header;<br>
> MyProtocolBody body;<br>
><br>
> static class MyProtocolHeader {......}<br>
> static class MyProtocolBody {......}<br>
><br>
> }<br>
><br>
> you should do the reverse process in MyProtocolEncoder<br>
><br>
> Hope it helps.<br>
><br>
> good luck<br>
><br>
> yanky<br>
><br>
><br>
> 2009/4/13 manishr22 <<a href="mailto:manishr22@gmail.com">manishr22@gmail.com</a>><br>
><br>
> ><br>
> > Hi,<br>
> ><br>
> > How I can read a chunk of data from channel using netty api? my client is<br>
> > writing every time on socket<br>
> ><br>
> > <manish> welcome to the real world</manish><br>
> ><br>
> > and my netty server needs to read the message till </manish> in one go.<br>
> and<br>
> > then next message starting from <manish> should be read by Netty server.<br>
> ><br>
> ><br>
> > Any code snippet would be a great help.<br>
> ><br>
> > In my decoder if i read the message from channelbuffer using getbytes()<br>
> api<br>
> > i see the data but not able to get i can read the whole message in one<br>
> go. i<br>
> > do see sometime some extra message being read from channel. any reason<br>
> why<br>
> > it should be.<br>
> ><br>
> ><br>
> ><br>
> > Thanks in advance.<br>
> > Regards<br>
> > Manish<br>
> ><br>
> > Hi Yanky,<br>
> ><br>
> > Thanks so much for your guidance. I will look into factorial example will<br>
> > solving my problem based on this.<br>
> ><br>
> > For any further help, if needed i will post my request here.<br>
> ><br>
> > Thanks again<br>
> ><br>
> > Manish<br>
> ><br>
> ><br>
> ><br>
> > Thanks Yanky for the great answer! So nice that you are here as a<br>
> > part of the community.<br>
> ><br>
> > Cheers,<br>
> ><br>
> > — Trustin Lee, <a href="http://gleamynode.net/" target="_blank">http://gleamynode.net/</a><br>
> ><br>
> > On Sat, Apr 11, 2009 at 3:16 AM, yanky young <<a href="mailto:yanky.young@gmail.com">yanky.young@gmail.com</a>><br>
> > wrote:<br>
> > > Hi:<br>
> > ><br>
> > > I am not netty expert yet. I am just trying to give some hints. In your<br>
> > > case, you are actually building a request/response protocol. And<br>
> request<br>
> > and<br>
> > > response message may include quite complicated headers and body. So you<br>
> > have<br>
> > > to do some message encoding and decoding in both server side and client<br>
> > > side. You can see in netty codebase there are some out-of-box encoder<br>
> and<br>
> > > decoder in handler.codec package. I suggest you take a look at<br>
> > > example.fractorial package to see how to implement your own encoder and<br>
> > > decoder for your message format which may be xml. If you are not<br>
> > sensitive<br>
> > > to message format, you can also have a look at example.localtime which<br>
> > use<br>
> > > google protobuf library as real encoder and decoder.<br>
> > ><br>
> > > good luck<br>
> > ><br>
> > > yanky<br>
> > ><br>
> > ><br>
> > ><br>
> > > 2009/4/11 manishr22 <<a href="mailto:manishr22@gmail.com">manishr22@gmail.com</a>><br>
> > >><br>
> > >> Hi Yanky,<br>
> > >><br>
> > >> I took the sample code from svn trunk and saw the codes too. Could you<br>
> > >> please let me know which one among given sample code will fit for the<br>
> > >> reference of my requirement.<br>
> > >><br>
> > >> Regards,<br>
> > >> Manish<br>
> > >><br>
> > >> Hi:<br>
> > >><br>
> > >> Of course there are some sample code in netty project. You can just<br>
> > check<br>
> > >> out the svn trunk and see some samples in example package.<br>
> > >><br>
> > >> good luck<br>
> > >><br>
> > >> yanky<br>
> > >><br>
> > >><br>
> > >> 2009/4/10 manishr22 <<a href="mailto:manishr22@gmail.com">manishr22@gmail.com</a>><br>
> > >><br>
> > >> ><br>
> > >> > Hi,<br>
> > >> ><br>
> > >> > I had a problem statement<br>
> > >> ><br>
> > >> > "to write an asynchronous TCP server to which multiple clients will<br>
> > >> > connect<br>
> > >> > and keep sending messages to my tcp server. My server should read<br>
> the<br>
> > >> > message and send back some acknowledgment information to the client<br>
> > who<br>
> > >> > sent<br>
> > >> > the data. While reading the data at server since the message comes<br>
> in<br>
> > >> > particular format i have to do some processing before sending the<br>
> > >> > response<br>
> > >> > back to client."<br>
> > >> ><br>
> > >> > say my client is sending a message structure in which it has some<br>
> > header<br>
> > >> > and then in body it has message like :<br>
> > >> ><br>
> > >> > <manish>welcome to the real world</manish><br>
> > >> ><br>
> > >> > I was on the way to implement NIO but in between i found such a<br>
> great<br>
> > >> > library netty and thought of using it. i was trying to make use of<br>
> > this<br>
> > >> > library with the given tutorial and examples but i found that<br>
> tutorial<br>
> > >> > is<br>
> > >> > not good enough to solve the complex problem.<br>
> > >> ><br>
> > >> > If you can provide me some guidance or sample code that how i should<br>
> > >> > solve<br>
> > >> > my problem using netty library, that would be great.<br>
> > >> ><br>
> > >> > Thanks in advance.<br>
> > >> ><br>
> > >> > Regards<br>
> > >> > Manish<br>
> > >> > --<br>
> > >> > View this message in context:<br>
> > >> ><br>
> > >> ><br>
> ><br>
> <a href="http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2615889.html" target="_blank">http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2615889.html</a><br>
> > >> > Sent from the Netty User Group mailing list archive at Nabble.com.<br>
> > >> ><br>
> > >> > _______________________________________________<br>
> > >> > netty-users mailing list<br>
> > >> > <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> > >> > <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
> > >> ><br>
> > >><br>
> > >> _______________________________________________<br>
> > >> netty-users mailing list<br>
> > >> <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> > >> <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
> > >><br>
> > >><br>
> > >><br>
> > >> --<br>
> > >> View this message in context:<br>
> > >><br>
> ><br>
> <a href="http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2617267.html" target="_blank">http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2617267.html</a><br>
> > >> Sent from the Netty User Group mailing list archive at Nabble.com.<br>
> > >><br>
> > >> _______________________________________________<br>
> > >> netty-users mailing list<br>
> > >> <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> > >> <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
> > ><br>
> > ><br>
> > > _______________________________________________<br>
> > > netty-users mailing list<br>
> > > <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> > > <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
> > ><br>
> > ><br>
> ><br>
> > _______________________________________________<br>
> > netty-users mailing list<br>
> > <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> > <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > View this message in context:<br>
> ><br>
> <a href="http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2627351.html" target="_blank">http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2627351.html</a><br>
> > Sent from the Netty User Group mailing list archive at Nabble.com.<br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > netty-users mailing list<br>
> > <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> > <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
> ><br>
><br>
> _______________________________________________<br>
> netty-users mailing list<br>
> <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
><br>
><br>
><br>
> --<br>
> View this message in context:<br>
> <a href="http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2632783.html" target="_blank">http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2632783.html</a><br>
> Sent from the Netty User Group mailing list archive at Nabble.com.<br>
><br>
><br>
> _______________________________________________<br>
> netty-users mailing list<br>
> <a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
><br>
<br>
_______________________________________________<br>
netty-users mailing list<br>
<a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
<br>
<br>
<br>
<br>
<br>
--<br>
</div></div>View this message in context: <a href="http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2642120.html" target="_blank">http://n2.nabble.com/Code-example-to-write-asynchronous-server-tp2615889p2642120.html</a><br>
<div><div></div><div class="h5">Sent from the Netty User Group mailing list archive at Nabble.com.<br>
<br>
<br>
_______________________________________________<br>
netty-users mailing list<br>
<a href="mailto:netty-users@lists.jboss.org">netty-users@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/netty-users" target="_blank">https://lists.jboss.org/mailman/listinfo/netty-users</a><br>
</div></div></blockquote></div><br>