[jboss-jira] [JBoss JIRA] (JGRP-1672) Shared memory to send message between differen processes on the same box

[Bela Ban (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796428#comment-12796428 ] 

Bela Ban commented on JGRP-1672:
--------------------------------

I'll be looking into using shared memory for passing messages between nodes on the same box in 3.5, and I listed some of the issues above. Caveat: "looking into" != "implementing" ...

This only makes sense anyway if we use direct memory (ByteBuffer and subclasses), or else we'd have one copy, so 3.5 and its use of NIO2 is a requirement.

If you're looking at using the Infinispan:JGroups combo, then the lower hanging fruit are probably Infinispan using an out-of-Java-heap container, possibly allocated in shared memory, so changes to it are visible to all nodes running on the same box. This would then make (JGroups) communication via shared memory moot.

Also, if RDMA comes into play, and you're running on Linux or Solaris, then SDP should have a big effect, not only for communication to remote nodes, but also to co-located nodes. I wonder what the diff between using shared memory and RDMA is for communication between nodes on the same box...

                
> Shared memory to send message between differen processes on the same box
> ------------------------------------------------------------------------
>
>                 Key: JGRP-1672
>                 URL: https://issues.jboss.org/browse/JGRP-1672
>             Project: JGroups
>          Issue Type: Feature Request
>            Reporter: Bela Ban
>            Assignee: Bela Ban
>             Fix For: 3.5
>
>
> Investigate whether it makes sense to use shared memory to pass messages between processes on the same box. Say if we have A, B and C on box-1 and X, Y, Z on box-2, when A multicasts a message, it could loop it back to itself, place it into shared memory for B and C to read and multicast it to X, Y, Z. The multicast socket could be non-loopback, so box-1 would not receive it.
> Problems:
> * Shared memory in Java can only be done via memory mapped (sequential or random access) files. To pass a lot of messages, something like a ring buffer would have to be created in shared memory
> * Unless we use FileLock, or polling/busy reading, there is no way to know when a producer has written a message into shared memory. We'd therefore have to use a signalling mechanism, probably a small JGroups message, to notify the consumer(s) of new messages
> * For multicast messages, we'd have 1 producer but many consumers. A RingBuffer would not work here, as we don't know when all consumers have read a given message, ie. when to advance the read pointer
> ** As an alternative, we could have one shared memory buffer per member on the same host. This would also cater to unicast messages. However, then we'd use up a lot of memory.
> * How would this work for TCP ? We'd have to send the message to only members which are outside the local box. How do we identify those members ?
> * Message reception: a multicast message received and targetted to all members on the same box could also be placed into shared memory, so everyone on the same box receives it
> ** How would this work for TCP ? E.g. A sending a multicast message M would use shared memory to deliver M to B and C on box-1, but if it sends it to X, Y and Z, then that's unneeded work, as it could send it only to X, which could place it into shared memory for Y and Z to consume M.
> *** We'd have to include the knowledge of 'affinity' into an address

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira