[
https://issues.jboss.org/browse/JGRP-1672?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1672:
--------------------------------
I'll be looking into using shared memory for passing messages between nodes on the
same box in 3.5, and I listed some of the issues above. Caveat: "looking into"
!= "implementing" ...
This only makes sense anyway if we use direct memory (ByteBuffer and subclasses), or else
we'd have one copy, so 3.5 and its use of NIO2 is a requirement.
If you're looking at using the Infinispan:JGroups combo, then the lower hanging fruit
are probably Infinispan using an out-of-Java-heap container, possibly allocated in shared
memory, so changes to it are visible to all nodes running on the same box. This would then
make (JGroups) communication via shared memory moot.
Also, if RDMA comes into play, and you're running on Linux or Solaris, then SDP should
have a big effect, not only for communication to remote nodes, but also to co-located
nodes. I wonder what the diff between using shared memory and RDMA is for communication
between nodes on the same box...
Shared memory to send message between differen processes on the same
box
------------------------------------------------------------------------
Key: JGRP-1672
URL:
https://issues.jboss.org/browse/JGRP-1672
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 3.5
Investigate whether it makes sense to use shared memory to pass messages between
processes on the same box. Say if we have A, B and C on box-1 and X, Y, Z on box-2, when A
multicasts a message, it could loop it back to itself, place it into shared memory for B
and C to read and multicast it to X, Y, Z. The multicast socket could be non-loopback, so
box-1 would not receive it.
Problems:
* Shared memory in Java can only be done via memory mapped (sequential or random access)
files. To pass a lot of messages, something like a ring buffer would have to be created in
shared memory
* Unless we use FileLock, or polling/busy reading, there is no way to know when a
producer has written a message into shared memory. We'd therefore have to use a
signalling mechanism, probably a small JGroups message, to notify the consumer(s) of new
messages
* For multicast messages, we'd have 1 producer but many consumers. A RingBuffer would
not work here, as we don't know when all consumers have read a given message, ie. when
to advance the read pointer
** As an alternative, we could have one shared memory buffer per member on the same host.
This would also cater to unicast messages. However, then we'd use up a lot of memory.
* How would this work for TCP ? We'd have to send the message to only members which
are outside the local box. How do we identify those members ?
* Message reception: a multicast message received and targetted to all members on the
same box could also be placed into shared memory, so everyone on the same box receives it
** How would this work for TCP ? E.g. A sending a multicast message M would use shared
memory to deliver M to B and C on box-1, but if it sends it to X, Y and Z, then that's
unneeded work, as it could send it only to X, which could place it into shared memory for
Y and Z to consume M.
*** We'd have to include the knowledge of 'affinity' into an address
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:
http://www.atlassian.com/software/jira