[jboss-jira] [JBoss JIRA] Commented: (JGRP-1090) Many retransmissions with message bundling and OOB messages

[Bela Ban (JIRA)]

    [ https://jira.jboss.org/jira/browse/JGRP-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12492993#action_12492993 ] 

Bela Ban commented on JGRP-1090:
--------------------------------

Idea to think about: submit each message in a bundle individually to the thread pool:

#1 A thread sequentially passes all messages of a bundle up the stack
- If a bundle 1 [100-120] is processed, thread A passes 100 up first, then 101 and so on. Thread A currently removes and processes 100
- Meanwhile thread B processes bundle 2 [121-130], adds all messages into the NRW. This causes retransmission requests for [101-120]
- When thread A returns from processing 100, it adds 101 to the NRW, cancelling the retransmission of 101, then 102 etc
- The issue is that processing 101-120 takes time, during which retransmission for 101-120 will be going on

A better solution would be if thread A added *all* messages [100-120] into the NRW *before* it started removing them. This would work with submitting each message of a bundle to the thread pool, e.g.

if(is_message_list) { // used if message bundling is enabled
    List<Message> msgs=readMessageList(dis);
    for(Message msg: msgs) {
        if(msg.isFlagSet(Message.OOB)) {
           log.warn("bundled message should not be marked as OOB");
        }
                      
    pool.execute(new Runnable() {public void run() {handleMyMessage(msg, multicast);}}
   
}


However, this taxes the thread pool, creates more threads and thus more context switching, which is also overhead. As a matter of fact, I tried this out and got a 5-10% performance drop.

> Many retransmissions with message bundling and OOB messages
> -----------------------------------------------------------
>
>                 Key: JGRP-1090
>                 URL: https://jira.jboss.org/jira/browse/JGRP-1090
>             Project: JGroups
>          Issue Type: Task
>            Reporter: Bela Ban
>            Assignee: Bela Ban
>             Fix For: 2.9
>
>
> When message bundling is enabled, we queue *regular* messages until max_bundle_size has been reached or max_bundle_timeout has exceeded.
> The problem is that - when sending OOB (as opposed to regular) messages - they're *not* queued. This means the OOB message is possibly received *before* the regular messages and thus causes retransmission !
> E.g. we send regular message 1-10, then OOB message 11.
> Say 1-3 are sent immediately (because the bundle timeout kicked in or the size was exceeded). Messages 4-10 are queued and sent later. Message 11 is sent immediately.
> On the receiver side, message reception could be for example: 1-3, 11, 4-10
> When OOB message 11 is received, the receiver asks for retransmission of messages 4-10 !
> This is not incorrect, but causes a lot of unneeded retransmissions !
>   public static void main(String[] args) throws Exception {
>         JChannel c1=new JChannel("/home/bela/fast.xml");
>         c1.connect("bla3");
>         JChannel c2=new JChannel("/home/bela/fast.xml");
>         c2.connect("bla3");
>         for(int i=1; i <= 20; i++) {
>             c1.send(null, null, "hello-" + i);
>         }
>         Message oob_msg=new Message(null, null, "oob msg");
>         oob_msg.setFlag(Message.OOB);
>         c1.send(oob_msg);
>         Util.sleep(20000);
>         Util.close(c2, c1);
>     }

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira