<div dir="ltr">On Fri, Aug 19, 2016 at 10:18 AM, Bela Ban <span dir="ltr"><<a href="mailto:bban@redhat.com" target="_blank">bban@redhat.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Sebastian<br>
<br>
the usual restrictions apply: if DNS discovery depends on external libs,<br>
then it should be hosted in jgroups-extras, otherwise we can add it to<br>
JGroups itself.<br></blockquote><div><br><br></div><div>Bela, you forgot to mention an extra restriction: even if it has no external deps, it should<br></div><div>not be too big in terms of LOC (what is the threshold?) ;)<br><br></div><div>Gustavo <br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class=""><br>
On 19/08/16 11:00, Sebastian Laskawiec wrote:<br>
> Hey!<br>
><br>
> I've been playing with Kubernetes PetSets [1] for a while and I'd like<br>
> to share some thoughts. Before I dig in, let me give you some PetSets<br>
> highlights:<br>
><br>
</span>> * PetSets are alpha resources for managing stateful apps in Kubernetes<br>
<span class="">> 1.3 (and OpenShift Origin 1.3).<br>
</span>> * Since this is an alpha resource, there are no guarantees about<br>
<span class="">> backwards compatibility. Alpha resources can also be disabled in<br>
> some public cloud providers (you can control which API versions are<br>
> accessible [2]).<br>
</span>> * PetSets allows starting pods in sequence (not relevant for us, but<br>
<span class="">> this is a killer feature for master-slave systems).<br>
</span>> * Each Pod has it's own unique entry in DNS, which makes discovery<br>
<span class="">> very simple (I'll dig into that a bit later)<br>
</span>> * Volumes are always mounted to the same Pods, which is very important<br>
<span class="">> in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades<br>
> [3]).<br>
><br>
> Thoughts and ideas after spending some time playing with this feature:<br>
><br>
</span>> * PetSets make discovery a lot easier. It's a combination of two<br>
<span class="">> things - Headless Services [4] which create multiple A records in<br>
> DNS and predictable host names. Each Pod has it's own unique DNS<br>
> entry following pattern: {PetSetName}-{PodIndex}.{<wbr>ServiceName} [5].<br>
> Here's an example of an Infinispan PetSet deployed on my local<br>
> cluster [6]. As you can see we have all domain names and IPs from a<br>
> single DNS query.<br>
</span>> * Maybe we could perform discovery using this mechanism? I'm aware of<br>
<span class="">> DNS discovery implemented in KUBE_PING [7][8] but the code looks<br>
> trivial [9] so maybe it should be implement inside JGroups? @Bela -<br>
> WDYT?<br>
</span>> * PetSets do not integrate well with OpenShift 'new-app' command. In<br>
<span class="">> other words, our users will need to use provided yaml (or json)<br>
> files to create Infinispan cluster. It's not a show-stopper but it's<br>
> a bit less convenient than 'oc new-app'.<br>
</span>> * Since PetSets are alpha resources they need to be considered as<br>
<span class="">> secondary way to deploy Infinispan on Kubernetes and OpenShift.<br>
</span>> * Finally, the persistent volumes - since a Pod always gets the same<br>
<div class="HOEnZb"><div class="h5">> volume, it would be safe to use any file-based cache store.<br>
><br>
> If you'd like to play with PetSets on your local environment, here are<br>
> necessary yaml files [10].<br>
><br>
> Thanks<br>
> Sebastian<br>
><br>
><br>
> [1] <a href="http://kubernetes.io/docs/user-guide/petset/" rel="noreferrer" target="_blank">http://kubernetes.io/docs/<wbr>user-guide/petset/</a><br>
> [2] For checking which APIs are accessible, use 'kubectl api-versions'<br>
> [3]<br>
> <a href="http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter" rel="noreferrer" target="_blank">http://infinispan.org/docs/<wbr>stable/user_guide/user_guide.<wbr>html#_Rolling_chapter</a><br>
> [4] <a href="http://kubernetes.io/docs/user-guide/services/#headless-services" rel="noreferrer" target="_blank">http://kubernetes.io/docs/<wbr>user-guide/services/#headless-<wbr>services</a><br>
> [5] <a href="http://kubernetes.io/docs/user-guide/petset/#peer-discovery" rel="noreferrer" target="_blank">http://kubernetes.io/docs/<wbr>user-guide/petset/#peer-<wbr>discovery</a><br>
> [6] <a href="https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676" rel="noreferrer" target="_blank">https://gist.github.com/<wbr>slaskawi/<wbr>0866e63a39276f8ab66376229716a6<wbr>76</a><br>
> [7] <a href="https://github.com/jboss-openshift/openshift-ping/tree/master/dns" rel="noreferrer" target="_blank">https://github.com/jboss-<wbr>openshift/openshift-ping/tree/<wbr>master/dns</a><br>
> [8] <a href="https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns" rel="noreferrer" target="_blank">https://github.com/jgroups-<wbr>extras/jgroups-kubernetes/<wbr>tree/master/dns</a><br>
> [9] <a href="http://stackoverflow.com/a/12405896/562699" rel="noreferrer" target="_blank">http://stackoverflow.com/a/<wbr>12405896/562699</a><br>
> [10] You might need to adjust ImageStream.<br>
> <a href="https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0" rel="noreferrer" target="_blank">https://gist.github.com/<wbr>slaskawi/<wbr>7cffb5588dabb770f654557579c5f2<wbr>d0</a><br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
Bela Ban, JGroups lead (<a href="http://www.jgroups.org" rel="noreferrer" target="_blank">http://www.jgroups.org</a>)<br>
<br>
______________________________<wbr>_________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/infinispan-<wbr>dev</a><br>
</font></span></blockquote></div><br></div></div>