<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 11, 2016 at 11:57 AM, Brian Stansberry <span dir="ltr"><<a href="mailto:brian.stansberry@redhat.com" target="_blank">brian.stansberry@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Just an FYI: I spent a couple days and worked up a POC[1] of creating a<br>
JGroups-based reliable group communication mesh over the sockets our<br>
Host Controllers use for intra-domain management communications.<br>
<br></blockquote><div><br></div><div>Nice! I've been thinking about the mechanics of this a bit recently, but I hadn't gotten to any sort of transport details, this looks interesting.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Currently those sockets are used to form a tree of connections; master<br>
HC to slave HCs and then HCs to their servers. Slave HCs don't talk to<br>
each other. That kind of topology works fine for our current use cases,<br>
but not for other use cases, where a full communication mesh is more<br>
appropriate.<br>
<br>
2 use cases led me to explore this:<br>
<br>
1) A longstanding request to have automatic failover of the master HC to<br>
a backup. There are different ways to do this, but group communication<br>
based leader election is a possible solution. My preference, really.<br></blockquote><div><br></div><div>I'd come to the same conclusion of it being an election. A deterministic election algorithm, perhaps allowing the configuration to supply some sort of weighted value to influence the election on each node, perhaps analogous to how the master browser smb election works (version + weight + etc).</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
2) <a href="https://issues.jboss.org/browse/WFLY-1066" rel="noreferrer" target="_blank">https://issues.jboss.org/browse/WFLY-1066</a>, which has led to various<br>
design alternatives, one of which is a distributed cache of topology<br>
information, available via each HC. See [2] for some of that discussion.<br>
<br>
I don't know if this kind of communication is a good idea, or if it's<br>
the right solution to either of these use cases. Lots of things need<br>
careful thought!! But I figured it was worth some time to experiment.<br>
And it worked in at least a basic POC way, hence this FYI.<br></blockquote><div><br></div><div>Not knowing a lot about jgroups .. for very large domains is the mesh NxN in size? For thousands of nodes would this become a problem, or would</div><div>a mechanism to segment into local groups perhaps, with only certain nodes participating in the mesh and being eligible for election?</div><div> </div><div>Ken</div></div><br></div></div>