[infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets

[Sebastian Laskawiec]

Hey Kevin!

The timing for looking into PetSets is perfect I think. Kubernetes upstream
folks are thinking about Sticky IPs [16] (which are not essential for
Infinispan but other projects might be really interested in this) and I
asked some time ago about exposing PetSets to the outside world [17] (which
is essential for accessing the cluster using Hot Rod client).

Please keep me in the loop. I'm really interested in this.

Thanks
Sebastian


[16] https://github.com/kubernetes/kubernetes/issues/28969
[17] https://groups.google.com/forum/#!topic/kubernetes-dev/K-9KA_wMbmk

On Sun, Aug 21, 2016 at 7:05 PM, Kevin Conner <kconner at redhat.com> wrote:

> Apologies, I’ve been travelling to London for the JDG meeting.
>
> On 19 Aug 2016, at 10:00, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> > I've been playing with Kubernetes PetSets [1] for a while and I'd like
> to share some thoughts. Before I dig in, let me give you some PetSets
> highlights:
> >       • PetSets are alpha resources for managing stateful apps in
> Kubernetes 1.3 (and OpenShift Origin 1.3).
> >       • Since this is an alpha resource, there are no guarantees about
> backwards compatibility. Alpha resources can also be disabled in some
> public cloud providers (you can control which API versions are accessible
> [2]).
> >       • PetSets allows starting pods in sequence (not relevant for us,
> but this is a killer feature for master-slave systems).
> >       • Each Pod has it's own unique entry in DNS, which makes discovery
> very simple (I'll dig into that a bit later)
> >       • Volumes are always mounted to the same Pods, which is very
> important in Cache Store scenarios when we restart pods (e.g. Rolling
> Upgrades [3]).
> > Thoughts and ideas after spending some time playing with this feature:
> >       • PetSets make discovery a lot easier. It's a combination of two
> things - Headless Services [4] which create multiple A records in DNS and
> predictable host names. Each Pod has it's own unique DNS entry following
> pattern: {PetSetName}-{PodIndex}.{ServiceName} [5]. Here's an example of
> an Infinispan PetSet deployed on my local cluster [6]. As you can see we
> have all domain names and IPs from a single DNS query.
> >       • Maybe we could perform discovery using this mechanism? I'm aware
> of DNS discovery implemented in KUBE_PING [7][8] but the code looks trivial
> [9] so maybe it should be implement inside JGroups? @Bela - WDYT?
> >       • PetSets do not integrate well with OpenShift 'new-app' command.
> In other words, our users will need to use provided yaml (or json) files to
> create Infinispan cluster. It's not a show-stopper but it's a bit less
> convenient than 'oc new-app'.
> >       • Since PetSets are alpha resources they need to be considered as
> secondary way to deploy Infinispan on Kubernetes and OpenShift.
> >       • Finally, the persistent volumes - since a Pod always gets the
> same volume, it would be safe to use any file-based cache store.
> > If you'd like to play with PetSets on your local environment, here are
> necessary yaml files [10].
>
> PetSets are still in extremely early stages and I am part of a working
> group that has recently formed to cover the introduction of this within the
> products, including driving any necessary changes back upstream.  There
> will be Cloud Enablement involvement in this effort and any adoption of
> petsets will likely go through the existing project.
>
>         Kev
>
> --
> JBoss by Red Hat
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/60d070db/attachment.html