[JBoss JIRA] Created: (JGRP-1154) FC: separate implementations for multicast and unicast messages
by Bela Ban (JIRA)
FC: separate implementations for multicast and unicast messages
---------------------------------------------------------------
Key: JGRP-1154
URL: https://jira.jboss.org/jira/browse/JGRP-1154
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.11
Currently, FC handles both multicast (=group) and unicast messages. This has a few disadvantages:
- Complexity
- Unicast messages are not dependent on multicast messages (and vice versa), so why should this be different for flow control ? E.g. why
should a multicast message block waiting for credits from X because a recent unicast message was sent to X ?
- If we have separate flow control protocols for multicasts and unicasts, we can selectively remove one or the other. This is currently
not possible
Design (see separate documents in ./doc/design):
FC_MCAST:
-----------------
(see whether some concepts can be borrowed from SFC)
Sender:
- Every sender has a credits AtomicLong variable
- When sending a message and the message length exceeds credits, the sender blocks (until it gets credits from all current group members)
- Else credits is decremented by the message length and the message is sent
- When credits are received from all current group members, credits is reset to the initial value and all blocked senders are unblocked
- When a new member joins, credits is also reset and all blocked senders are unblocked
Receiver:
- Maintains a hashmap of senders and credits (Map<Address,AtomicLong>)
- When a message is received from S, the credits for S are decremented by the message length
- If the remaining length is less than the threshold, new credits are sent to S and S's credits are reset to their initial value
- When a new member joins, all credits (including the newly joined member) are reset to the initial value
FC_UNICAST:
--------------------
Sender:
- A hashmap of addresses and credits is maintained: every receiver has associated credits
- When a message is sent to P, if P's credits minus the message length is less than 0, the sender blocks until it gets new credits from the receiver
- Else P's credits are decremented by the message length and the message is sent
Receiver:
-Same as for FC_MCAST
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months
[JBoss JIRA] Resolved: (JGRP-1038) FC2
by Bela Ban (JIRA)
[ https://jira.jboss.org/browse/JGRP-1038?page=com.atlassian.jira.plugin.sy... ]
Bela Ban resolved JGRP-1038.
----------------------------
Resolution: Won't Fix
> FC2
> ---
>
> Key: JGRP-1038
> URL: https://jira.jboss.org/browse/JGRP-1038
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 2.11
>
> Attachments: FC2.java, FCOverloadTest.java, Results.txt
>
>
> [Email Gray Watson]
> So we implemented our own FC2 because of problems with FC. I've been writing
> tests and trying (with some success) to reproduce the issues we faced.
> Initially when we were doing performance tests in our cluster of ~20 big-ish
> linux servers with UDP/FC, we saw severe performance tests with "decent"
> sized messages. Often we have large SQL responses which are in the 1+mb
> range and can be as high as 10+mb. I decided to write the attached
> FCOverloadTest to demonstrate some of the issues and to compare FC and FC2.
> The default FC setting max_credits=500000 in udp.xml results in terrible
> performance -- 952 seconds to send 10x 5mb messages to 10 hosts (500mb).
> Less than optimal.
> The simple solution, on the surface, is to increase the credits 1 or 2
> orders of magnitude. 5000000 takes 160 secs, and 50000000 takes 32 secs.
> Fine. But this in effect removes most of the flow-control. The larger the
> max-credits, the more messages queue up.
> What I thought would be a better flow-control model was per-recipient, where
> each sender would keep a "water-level" of the outstanding packets needed to
> be ack'd. When the water level got too high it would pause the sender until
> the receiver had ack'd the water level back down. This is what FC2 tried to
> accomplish. I've attached my test results, the FCOverloadTest, and FC2.
> FC2 looks to do less UNICAST retransmits, sends fewer packets, and is
> faster. It needs some review however.
> I'd like to know other folks' experiences with flow-control. Are people
> really using the udp.xml settings in production? What message sizes do you
> have?
> The next step in my testing is to write some unit tests which shut down and
> start up notes while the system is loaded with an attempt to reproduce the
> cluster pauses with FC that we have seen which would be deadly to our
> application.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months
[JBoss JIRA] Commented: (JGRP-1038) FC2
by Bela Ban (JIRA)
[ https://jira.jboss.org/browse/JGRP-1038?page=com.atlassian.jira.plugin.sy... ]
Bela Ban commented on JGRP-1038:
--------------------------------
Gray withdrew this contribution, as he tested with regular FC and got good enough perf.
> FC2
> ---
>
> Key: JGRP-1038
> URL: https://jira.jboss.org/browse/JGRP-1038
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 2.11
>
> Attachments: FC2.java, FCOverloadTest.java, Results.txt
>
>
> [Email Gray Watson]
> So we implemented our own FC2 because of problems with FC. I've been writing
> tests and trying (with some success) to reproduce the issues we faced.
> Initially when we were doing performance tests in our cluster of ~20 big-ish
> linux servers with UDP/FC, we saw severe performance tests with "decent"
> sized messages. Often we have large SQL responses which are in the 1+mb
> range and can be as high as 10+mb. I decided to write the attached
> FCOverloadTest to demonstrate some of the issues and to compare FC and FC2.
> The default FC setting max_credits=500000 in udp.xml results in terrible
> performance -- 952 seconds to send 10x 5mb messages to 10 hosts (500mb).
> Less than optimal.
> The simple solution, on the surface, is to increase the credits 1 or 2
> orders of magnitude. 5000000 takes 160 secs, and 50000000 takes 32 secs.
> Fine. But this in effect removes most of the flow-control. The larger the
> max-credits, the more messages queue up.
> What I thought would be a better flow-control model was per-recipient, where
> each sender would keep a "water-level" of the outstanding packets needed to
> be ack'd. When the water level got too high it would pause the sender until
> the receiver had ack'd the water level back down. This is what FC2 tried to
> accomplish. I've attached my test results, the FCOverloadTest, and FC2.
> FC2 looks to do less UNICAST retransmits, sends fewer packets, and is
> faster. It needs some review however.
> I'd like to know other folks' experiences with flow-control. Are people
> really using the udp.xml settings in production? What message sizes do you
> have?
> The next step in my testing is to write some unit tests which shut down and
> start up notes while the system is loaded with an attempt to reproduce the
> cluster pauses with FC that we have seen which would be deadly to our
> application.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months
[JBoss JIRA] Commented: (JBAS-7818) RMIAdaptor replacement based on JSR160
by Shelly McGowan (JIRA)
[ https://jira.jboss.org/browse/JBAS-7818?page=com.atlassian.jira.plugin.sy... ]
Shelly McGowan commented on JBAS-7818:
--------------------------------------
Scott, is this work completed for AS 6.0.0?
There is a problem starting AS 6 with IPv6; specifically, the error shown below. This prevents additional testing with IPv6 to proceed. Can you comment on this failure?
4:15:34,374 INFO [JMXConnector] starting JMXConnector on host 3ffe:ffff:100:f101::1:1090
14:15:34,393 ERROR [JMXConnector] Could not start JMXConnector: java.net.MalformedURLException: Bad port number: "": java.lang.NumberFormatException: For input string: ""
at javax.management.remote.JMXServiceURL.<init>(JMXServiceURL.java:193) [:1.6.0_13]
at org.jboss.system.server.jmx.JMXConnector.start(JMXConnector.java:143) [:6.0.0.20100721-M4 (Build SVNTag:JBoss_6.0.0.20100721-M4 date: 20100723)]
> RMIAdaptor replacement based on JSR160
> --------------------------------------
>
> Key: JBAS-7818
> URL: https://jira.jboss.org/browse/JBAS-7818
> Project: JBoss Application Server
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: JMX
> Reporter: Scott M Stark
> Assignee: Scott Marlow
> Fix For: TBD-6.x
>
>
> A replacement for the proprietary RMIAdaptor is needed that supports authentication, authorization and SSL.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months