This is fixed by minor configuration change.
In deploy\messaging\remoting-bisocket-service.xml add the following configuration
<attribute name="enableTcpNoDelay" isParam="true">true</attribute>
Above configuration had improved the performance.
Since we are using tcp channel instead of udp, we also modified
\deploy\cluster\jgroups-channelfactory.sar\META-INF\jgroups-channelfactory-stacks.xml
by adding below configuration to tcp and jbm-data channels
tcp_nodelay="true"
Both the above configurations helped the message delivery happening in sub-milliseconds time for the sample application which was consistently producing nearly 200ms time delay for 5% of messages.
I would like to hear from expert on any side effects of above configuration especially since Redhat has not documented the use of "enableTcpNoDelay" in JBoss 5.x version documentation.
Thanks,
Sudeep