[Design of Messaging on JBoss (Messaging/JBoss)] - Re: Good article on consistent hashing
by ataylor
I think we agree that using a consistent hashing approach will not work.
So I think some sort of consensus algorithm is the way to go.
Ive looked at the paxos algorithm which basically allows proposers to propose values creating a list of proposed values and then each node selecting the same proposed value. Since we always want to propose the local node if possible I've come up with the following consensus algorithm which i think will work.
when a message contains a group id the following happens before routing to a queue in BindingsImpl.
If there is only a local queue then the message is delivered to the local queue and bound. This basically means we don't have a clustered queue.
If there are remote queues and we haven't yet bound to any queue then the proposing node will propose its own local queue and send a propose message to each other node. Each node will return an accept message back to the node keeping a record of the groupid and the outcome. This is the simplest scenario where only one node receives messages with a particular group id and becomes bound to the group id.
If one of the other nodes has previously received a proposal for that groupid then it will decline the proposal. Basically this means that 1 or more nodes are vying to obtain the group binding. In the simplest scenario here 1 of the vying nodes will receive a majority of accepts and becomes bound to the group id.
If one of the other nodes has already bound to that groupid or it is routing messages to another node bound with this group id then it will also decline. The decline will also notify the proposer of the correct node to use. The proposer will then give up and route to this node.
Once the proposing node has received enough accepts (a majority of the other nodes, no of other nodes/2 + 1) then it will assign itself as the node for that groupid and send an accepted message to each other node.
If a node recieves a majority of declines then it will either receive a decline containing the node to forward to or an accepted message with the same info.
There is a possibility that if 2 or more nodes send a proposal at the same time and both decline the other that a stalemate may occur. consider the following:
N1 send proposal to N2
N1 send proposal to N3
N2 send proposal to N1
N2 send proposal to N2
N3 sends accept to N1
N3 sends decline to N2
N3 sends decline to N1
N1 sends decline to N2
at this point N2 knows it doesn't have the consensus however it has already sent a decline to N1. Since N1 only has 50% consensus it still doesn't have the majority. What we do here is to allow N2 to change its mind. It knows it has already declined N1 so at the point where it has received enough declines it sends an accept message to N1. N1 then has the consensus. so this would be:
N1 send proposal to N2
N1 send proposal to N3
N2 send proposal to N1
N2 send proposal to N2
N3 sends accept to N1
N3 sends decline to N2
N3 sends decline to N1
N1 sends decline to N2
N2 has 2 declines do sends an accept to N1
N1 binds to the group and sends an accepted message
This however only works when there are an odd number of nodes. If there were 4 nodes it would be possible that N1 and N2 both have 50% accepts and 50% declines. If this happens then we again use a consistent algorithm to decide. so this would be:
N1 send proposal to N2
N1 send proposal to N3
N1 send proposal to N4
N2 send proposal to N1
N2 send proposal to N2
N2 send proposal to N4
N3 sends accept to N1
N3 sends decline to N2
N4 sends accept to N2
N4 sends decline to N1
N3 sends decline to N1
N1 sends decline to N2
N1 and N2 boh have 1 accept so the hash the nodeid and send to the smallest/greatest values node, or some similar simple alg
N1 binds to the group and sends an accepted message
If as new node comes online then it does not need to know anything until it receives a message with the same group id, at this point it will propose itself but receive 1 or more declines informing it of the node to use.
If a node disappears the sending node will at that point try to propose itself as a new node and we reach for a new consensus.
There is also an issue where there may be a window where not a node is only known to a subset of the other nodes. This could happen when a node first comes up and it hasn't yet notified all the nodes in the cluster. In this scenario we may have to force the nodes only to use proposers from nodes known to every other node in the cluster. This is something I will need to look at but is an edge case and i will leave until later.
thoughts?
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4241076#4241076
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4241076
14 years, 9 months
[Design of JBossXB] - Re: JAXB annotations for OSGi Blueprint
by alex.loubyansky＠jboss.com
Richard, all these are non-problems.
"richard.opalka(a)jboss.com" wrote :
| * have the full control over the XML->Java binding process
| * have the full control over the code to do architectural changes
| * have the full control of X2JB project lifecycle (to fix/release immediately)
|
| I don't have that kind of control over the JAXB or XMLBeans thus I'm not going to use
| JBossXB, JAXB, XMLBeans and other binding frameworks neither now nor in the future
| in my not JBoss related projects.
You can have all of that. I don't think anybody would mind.
There are lots of custom binding features in XB. You are welcome to add more.
There have never been a complaint about project lifecycle for the reason that there is no roadmap. It's demand-driven. If someone needs a release we can do it often the same day or whenever it's needed.
You want to do architectural changes - really no problem. Given that the existing users are not affected of course. I guess that's the real thing you don't want to deal with.
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4241074#4241074
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4241074
14 years, 9 months
[Design of JBossXB] - Re: JAXB annotations for OSGi Blueprint
by richard.opalka＠jboss.com
"richard.opalka(a)jboss.com" wrote : "adrian(a)jboss.org" wrote :
| | don't write another, that's just
| | NIH syndrome.
| This is relative (like everything in this world). My point of view on NIH is nice expressed here.
| I don't consider MC as NIH comparing to Spring, because it's critical project for JBoss/RH.
| For the same reason I don't consider X2JB NIH because its critical for my projects because I need to:
|
| * have the full control over the XML->Java binding process
| * have the full control over the code to do architectural changes
| * have the full control of X2JB project lifecycle (to fix/release immediately)
|
| I don't have that kind of control over the JAXB nor XMLBeans thus I'm not going to use
| JBossXB, JAXB, XMLBeans and other binding frameworks neither now nor in the future.
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4241053#4241053
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4241053
14 years, 9 months
[Design of JBossXB] - Re: JAXB annotations for OSGi Blueprint
by richard.opalka＠jboss.com
"adrian(a)jboss.org" wrote : "thomas.diesler(a)jboss.com" wrote :
| | don't write another, that's just
| | NIH syndrome.
| This is relative (like everything in this world). My point of view on NIH is nice expressed here.
| I don't consider MC as NIH comparing to Spring, because it's critical project for JBoss/RH.
| For the same reason I don't consider X2JB NIH because its critical for my projects because I need to:
|
| * have the full control over the XML->Java binding process
| * have the full control over the code to do architectural changes
| * have the full control of X2JB project lifecycle (to fix/release immediately)
|
| I don't have that kind of control over the JAXB nor XMLBeans thus I'm not going to use
| JBossXB, JAXB, XMLBeans and other binding frameworks neither now nor in the future.
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4241052#4241052
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4241052
14 years, 9 months