[jboss-jira] [JBoss JIRA] (JGRP-1961) TCP_NIO2: message bundling can lead to corruption of msgs sent asynchronously

[Bela Ban (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105942#comment-13105942 ] 

Bela Ban edited comment on JGRP-1961 at 9/6/15 11:00 AM:
---------------------------------------------------------

For non-bundled messages, we create a new buffer for every message: those buffers are not reused. In contrast, message bundlers create a buffer (initial size: 1024 bytes) which is always reused and thus potentially increases.
Perhaps we need to abandon this and create a new buffer in bundlers for every message, too, like for non bundled messages...
Downside: this affects {{TCP}} and {{UDP}} which don't need to copy. 
Is blindly copying in {{TCP_PING}} an option ? The copies would be unneeded for non-bundled messages, but I guess most messages are bundled, so this should be ok...
Copying in {{TCP_NIO2}} is similar to copying in {{TCP}}, with send-queues being enabled...


was (Author: belaban):
For non-bundled messages, we create a new buffer for every message: those buffers are not reused. In contrast, message bundlers create a buffer (initial size: 1024 bytes) which is always reused and thus potentially increases.
Perhaps we need to abandon this and create a new buffer in bundlers for every message, too, like for non bundled messages...
Downside: this affects {{TCP}} and {{UDP}} which don't need to copy. 
Is blindly copying in {{TCP_PING}} an option ? The copies would be unneeded for non-bundled messages, but I guess most messages are bundled, so this should be ok...

> TCP_NIO2: message bundling can lead to corruption of msgs sent asynchronously
> -----------------------------------------------------------------------------
>
>                 Key: JGRP-1961
>                 URL: https://issues.jboss.org/browse/JGRP-1961
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 3.6.5
>            Reporter: Bela Ban
>            Assignee: Bela Ban
>            Priority: Critical
>             Fix For: 3.6.6
>
>
> When a message is sent with a message bundler (in {{TP}}), the output buffer in {{TP.BaseBundler}} is reused for all messages of a given bundle (say {{m1}} and {{m2}}).
> This works for {{TCP}} because a write _blocks_ and - when it returns - we're guaranteed that the buffer has been copied into an OS buffer (TCP's send window).
> However, this is different with {{TCP_NIO2}}: *an async write always returns immediately, irrespective of whether the data was written completely, partially, or not at all !*
> If a write only writes a part of its data, on returning from the write, the message bundler reuses the output buffer and can thus overwrite and corrupt buffers that are in transit, waiting to be written.
> A quick check where buffers were copied fixed the problem.
> TODO: see where we need to copy data when using a bundler. Possibly only copy if a write didn't write all of the data.



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)