[JBoss JIRA] (JGRP-1672) Shared memory to send message between different processes on the same box
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1672?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-1672:
---------------------------
Fix Version/s: 4.x
(was: 4.1)
> Shared memory to send message between different processes on the same box
> -------------------------------------------------------------------------
>
> Key: JGRP-1672
> URL: https://issues.jboss.org/browse/JGRP-1672
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 4.x
>
> Attachments: ShmTest.java
>
>
> Investigate whether it makes sense to use shared memory to pass messages between processes on the same box. Say if we have A, B and C on box-1 and X, Y, Z on box-2, when A multicasts a message, it could loop it back to itself, place it into shared memory for B and C to read and multicast it to X, Y, Z. The multicast socket could be non-loopback, so box-1 would not receive it.
> Problems:
> * Shared memory in Java can only be done via memory mapped (sequential or random access) files. To pass a lot of messages, something like a ring buffer would have to be created in shared memory
> * Unless we use FileLock, or polling/busy reading, there is no way to know when a producer has written a message into shared memory. We'd therefore have to use a signalling mechanism, probably a small JGroups message, to notify the consumer(s) of new messages.
> ** Alternatively, we could do busy waiting: the producer writes into a memory location when a message is ready to be consumed. Perhaps this memory location can be the number of messages ready to be read. The consumer could busy-wait, and decrement the number of messages read. This variable could be protected by a file lock, so after some amount of busy-waiting, the consumer could go back and do a real wait on the file lock, instead of burning CPU doing busy-waits.
> * For multicast messages, we'd have 1 producer but many consumers. A RingBuffer would not work here, as we don't know when all consumers have read a given message, ie. when to advance the read pointer
> ** As an alternative, we could have one shared memory buffer per member on the same host. This would also cater to unicast messages. However, then we'd use up a lot of memory.
> * How would this work for TCP ? We'd have to send the message to only members which are outside the local box. How do we identify those members ?
> * Message reception: a multicast message received and targetted to all members on the same box could also be placed into shared memory, so everyone on the same box receives it
> ** How would this work for TCP ? E.g. A sending a multicast message M would use shared memory to deliver M to B and C on box-1, but if it sends it to X, Y and Z, then that's unneeded work, as it could send it only to X, which could place it into shared memory for Y and Z to consume M.
> *** We'd have to include the knowledge of 'affinity' into an address
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 2 months
[JBoss JIRA] (JGRP-2065) RoundTrip: latency is high compared to RoundTripTcp/RoundTripServer
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2065?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-2065:
---------------------------
Fix Version/s: 4.x
(was: 4.1)
> RoundTrip: latency is high compared to RoundTripTcp/RoundTripServer
> -------------------------------------------------------------------
>
> Key: JGRP-2065
> URL: https://issues.jboss.org/browse/JGRP-2065
> Project: JGroups
> Issue Type: Task
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 4.x
>
>
> {{RoundTrip}} is a simple test between 2 members measuring round-trip latency. The sender continually sends a message, the receiver receives it and sends the response, and the sender unblocks when the response is received. Then the sender sends the next message.
> The time for the request is then logged at the sender and a min/avg/max value is computed (probably changed to histograms later).
> {{RoundTrip}} uses a JGroups channel, configured with {{-props}}, e.g. {{-props ~/tcp.xml}}.
> {{RoundTripTcp}} does the same, but uses direct TCP sockets (no JGroups) for communication.
> {{RoundTripServer}} uses the client-server classes of JGroups for communication, but no channel is used.
> Round trip times (both processes on the same box, a Mac mini):
> * {{RoundTrip}} (with {{tcp.xml}} shipped with JGroups): *110 us*
> * {{RoundTripTcp}}: *46 us*
> * {{RoundTripServer}}: *49 us*
> Note that the client/server classes used by {{RoundTripServer}} are also used by the TCP transport (configured in {{tcp.xml}}.
> {{RoundTripServer}} is ~6% slower than {{RoundTripTcp}}, but that can be attributed to the additional work the former has to do (e.g. connection reaping etc). This is something we can focus on later.
> The big difference are the 110 us for {{RoundTrip}}. The goal is to find out what causes this and eliminate it. Since {{RoundTrip}} and {{RoundTripServer}} use the same underlying client/server classes in JGroups, let's compare these 2.
> Tasks:
> * Remove all protocols other than TCP in the running stack: (e.g. {{probe.sh remove-protocol=MFC}}). I already did this and the diff was negligible, but let's run this again
> * Try various bundlers (e.g. NoBundler)
> * Reduce threads in the threadpools, possibly disable (regular and OOB) thread pools (replace with DirectExecutor)
> * The default is 1 sender thread, but try with multiple threads
> * Measure perf between sending a message (in the bundler) and receiving it
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 2 months
[JBoss JIRA] (JGRP-1424) TP: use of multiple transports
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1424?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-1424:
---------------------------
Fix Version/s: 4.x
(was: 4.1)
> TP: use of multiple transports
> ------------------------------
>
> Key: JGRP-1424
> URL: https://issues.jboss.org/browse/JGRP-1424
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 4.x
>
>
> Refactor TP so that the socket sending and receiving is done in a separate class (UDP, TCP, TCP_NIO). Once this is done, add the ability to attach multiple transports to TP, e.g. UDP and TCP.
> The UDP transport could then be used for cluster wide messages (null destination) and the TCP transport could be used for unicast messages (non-null destination).
> Or this could be overridden by a message flag on a per-message basis !
> We could even attach multiple transports of the same type, e.g. one per physical network (10.x.x.x and 192.168.x.x), and do round-robin sending over them.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 2 months
[JBoss JIRA] (JGRP-2140) Replace Table with RingBuffer in UNICAST3 and NAKACK2
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2140?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-2140:
---------------------------
Fix Version/s: 4.x
(was: 4.1)
> Replace Table with RingBuffer in UNICAST3 and NAKACK2
> -----------------------------------------------------
>
> Key: JGRP-2140
> URL: https://issues.jboss.org/browse/JGRP-2140
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 4.x
>
>
> Table does allocate memory on resizing or compaction; whereas RingBuffer doesn't. Replacing Table with RingBuffer might bring memory allocation rates even further down.
> However, using RingBuffer as message store means that the capacity is fixed; when the RingBuffer is full, messages would get dropped. This should not be a big issue as they will get retransmitted anyway, and flow control should actually kick in to throttle senders (this is done now, to prevent Table from growing out of bounds).
> Measure the impact on performance. Table is a critical class used by NAKACK2 and UNICAST3, and is battle tested. Do this only if the benefits trump the risk.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 2 months