<div dir="ltr">Hey Bela!<div><br></div><div>No no, the resolution can be done with pure JDK.</div><div><br></div><div>Thanks</div><div>Sebastian</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 19, 2016 at 11:18 AM, Bela Ban <span dir="ltr"><<a href="mailto:bban@redhat.com" target="_blank">bban@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Sebastian<br>
<br>
the usual restrictions apply: if DNS discovery depends on external libs, then it should be hosted in jgroups-extras, otherwise we can add it to JGroups itself.<span class=""><br>
<br>
On 19/08/16 11:00, Sebastian Laskawiec wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
Hey!<br>
<br>
I've been playing with Kubernetes PetSets [1] for a while and I'd like<br>
to share some thoughts. Before I dig in, let me give you some PetSets<br>
highlights:<br>
<br></span>
* PetSets are alpha resources for managing stateful apps in Kubernetes<span class=""><br>
1.3 (and OpenShift Origin 1.3).<br></span>
* Since this is an alpha resource, there are no guarantees about<span class=""><br>
backwards compatibility. Alpha resources can also be disabled in<br>
some public cloud providers (you can control which API versions are<br>
accessible [2]).<br></span>
* PetSets allows starting pods in sequence (not relevant for us, but<span class=""><br>
this is a killer feature for master-slave systems).<br></span>
* Each Pod has it's own unique entry in DNS, which makes discovery<span class=""><br>
very simple (I'll dig into that a bit later)<br></span>
* Volumes are always mounted to the same Pods, which is very important<span class=""><br>
in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades<br>
[3]).<br>
<br>
Thoughts and ideas after spending some time playing with this feature:<br>
<br></span>
* PetSets make discovery a lot easier. It's a combination of two<span class=""><br>
things - Headless Services [4] which create multiple A records in<br>
DNS and predictable host names. Each Pod has it's own unique DNS<br>
entry following pattern: {PetSetName}-{PodIndex}.{Servi<wbr>ceName} [5].<br>
Here's an example of an Infinispan PetSet deployed on my local<br>
cluster [6]. As you can see we have all domain names and IPs from a<br>
single DNS query.<br></span>
* Maybe we could perform discovery using this mechanism? I'm aware of<span class=""><br>
DNS discovery implemented in KUBE_PING [7][8] but the code looks<br>
trivial [9] so maybe it should be implement inside JGroups? @Bela -<br>
WDYT?<br></span>
* PetSets do not integrate well with OpenShift 'new-app' command. In<span class=""><br>
other words, our users will need to use provided yaml (or json)<br>
files to create Infinispan cluster. It's not a show-stopper but it's<br>
a bit less convenient than 'oc new-app'.<br></span>
* Since PetSets are alpha resources they need to be considered as<span class=""><br>
secondary way to deploy Infinispan on Kubernetes and OpenShift.<br></span>
* Finally, the persistent volumes - since a Pod always gets the same<span class=""><br>
volume, it would be safe to use any file-based cache store.<br>
<br>
If you'd like to play with PetSets on your local environment, here are<br>
necessary yaml files [10].<br>
<br>
Thanks<br>
Sebastian<br>
<br>
<br>
[1] <a href="http://kubernetes.io/docs/user-guide/petset/" rel="noreferrer" target="_blank">http://kubernetes.io/docs/user<wbr>-guide/petset/</a><br>
[2] For checking which APIs are accessible, use 'kubectl api-versions'<br>
[3]<br>
<a href="http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter" rel="noreferrer" target="_blank">http://infinispan.org/docs/sta<wbr>ble/user_guide/user_guide.html<wbr>#_Rolling_chapter</a><br>
[4] <a href="http://kubernetes.io/docs/user-guide/services/#headless-services" rel="noreferrer" target="_blank">http://kubernetes.io/docs/user<wbr>-guide/services/#headless-serv<wbr>ices</a><br>
[5] <a href="http://kubernetes.io/docs/user-guide/petset/#peer-discovery" rel="noreferrer" target="_blank">http://kubernetes.io/docs/user<wbr>-guide/petset/#peer-discovery</a><br>
[6] <a href="https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676" rel="noreferrer" target="_blank">https://gist.github.com/slaska<wbr>wi/0866e63a39276f8ab6637622971<wbr>6a676</a><br>
[7] <a href="https://github.com/jboss-openshift/openshift-ping/tree/master/dns" rel="noreferrer" target="_blank">https://github.com/jboss-opens<wbr>hift/openshift-ping/tree/maste<wbr>r/dns</a><br>
[8] <a href="https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns" rel="noreferrer" target="_blank">https://github.com/jgroups-ext<wbr>ras/jgroups-kubernetes/tree/<wbr>master/dns</a><br>
[9] <a href="http://stackoverflow.com/a/12405896/562699" rel="noreferrer" target="_blank">http://stackoverflow.com/a/124<wbr>05896/562699</a><br>
[10] You might need to adjust ImageStream.<br>
<a href="https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0" rel="noreferrer" target="_blank">https://gist.github.com/slaska<wbr>wi/7cffb5588dabb770f654557579c<wbr>5f2d0</a><br>
</span></blockquote><span class="HOEnZb"><font color="#888888">
<br>
-- <br>
Bela Ban, JGroups lead (<a href="http://www.jgroups.org" rel="noreferrer" target="_blank">http://www.jgroups.org</a>)<br>
<br>
</font></span></blockquote></div><br></div>