[infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
Sebastian Laskawiec
slaskawi at redhat.com
Wed Aug 24 03:07:35 EDT 2016
Hey Alan!
Yeah, that could work :) I'll be doing some experiments with volumes
shortly so I'll definitely check that!
Thanks
Sebastian
On Mon, Aug 22, 2016 at 8:14 PM, Alan Field <afield at redhat.com> wrote:
> A little late to the discussion but if all of the pods share the same set
> of volumes, couldn't you also use the FILE_PING discovery protocol between
> the pods?
>
> Thanks,
> Alan
>
> ------------------------------
>
> *From: *"Sebastian Laskawiec" <slaskawi at redhat.com>
> *To: *"infinispan -Dev List" <infinispan-dev at lists.jboss.org>, "Bela Ban"
> <bban at redhat.com>, "Kevin Conner" <kconner at redhat.com>, "Rob Cernich" <
> rcernich at redhat.com>
> *Sent: *Friday, August 19, 2016 5:00:12 AM
> *Subject: *[infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
>
>
> Hey!
>
> I've been playing with Kubernetes PetSets [1] for a while and I'd like to
> share some thoughts. Before I dig in, let me give you some PetSets
> highlights:
>
> - PetSets are alpha resources for managing stateful apps in Kubernetes
> 1.3 (and OpenShift Origin 1.3).
> - Since this is an alpha resource, there are no guarantees about
> backwards compatibility. Alpha resources can also be disabled in some
> public cloud providers (you can control which API versions are accessible
> [2]).
> - PetSets allows starting pods in sequence (not relevant for us, but
> this is a killer feature for master-slave systems).
> - Each Pod has it's own unique entry in DNS, which makes discovery
> very simple (I'll dig into that a bit later)
> - Volumes are always mounted to the same Pods, which is very important
> in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades [3]).
>
> Thoughts and ideas after spending some time playing with this feature:
>
> - PetSets make discovery a lot easier. It's a combination of two
> things - Headless Services [4] which create multiple A records in DNS and
> predictable host names. Each Pod has it's own unique DNS entry following
> pattern: {PetSetName}-{PodIndex}.{ServiceName} [5]. Here's an example
> of an Infinispan PetSet deployed on my local cluster [6]. As you can see we
> have all domain names and IPs from a single DNS query.
> - Maybe we could perform discovery using this mechanism? I'm aware of
> DNS discovery implemented in KUBE_PING [7][8] but the code looks trivial
> [9] so maybe it should be implement inside JGroups? @Bela - WDYT?
> - PetSets do not integrate well with OpenShift 'new-app' command. In
> other words, our users will need to use provided yaml (or json) files to
> create Infinispan cluster. It's not a show-stopper but it's a bit less
> convenient than 'oc new-app'.
> - Since PetSets are alpha resources they need to be considered as
> secondary way to deploy Infinispan on Kubernetes and OpenShift.
> - Finally, the persistent volumes - since a Pod always gets the same
> volume, it would be safe to use any file-based cache store.
>
> If you'd like to play with PetSets on your local environment, here are
> necessary yaml files [10].
>
> Thanks
> Sebastian
>
>
> [1] http://kubernetes.io/docs/user-guide/petset/
> [2] For checking which APIs are accessible, use 'kubectl api-versions'
> [3] http://infinispan.org/docs/stable/user_guide/user_
> guide.html#_Rolling_chapter
> [4] http://kubernetes.io/docs/user-guide/services/#headless-services
> [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
> [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
> [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
> [8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
> [9] http://stackoverflow.com/a/12405896/562699
> [10] You might need to adjust ImageStream. https://gist.github.com/
> slaskawi/7cffb5588dabb770f654557579c5f2d0
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160824/5585ede7/attachment-0001.html
More information about the infinispan-dev
mailing list