[infinispan-dev] KUBE_PING changes

Thomas SEGISMONT tsegismont at gmail.com
Mon Jul 3 03:37:12 EDT 2017 

Hi Sebastian,

2017-07-03 8:24 GMT+02:00 Sebastian Laskawiec <slaskawi at redhat.com>:

> Hey Thomas,
>
> Comments inlined.
>
> Thanks,
> Sebastian
>
>
> On Fri, Jun 30, 2017 at 9:29 PM Thomas SEGISMONT <tsegismont at gmail.com>
> wrote:
>
>> Also, it seems infinispan-cloud 9.0.3.Final depends on JGroups 0.9.1.
>>
>> Do you plan to release another 9.0.x version which depends on 1.0.0.Beta1
>> or later? If so, is there a target date?
>>
>
> No, I didn't plan to backport it to 9.0.x branch. The implementation is
> pretty new and I wanted to play with it a little bit more before make it
> "stable".
>
> Could you please tell me why do you need it?
>

On Openshift S2I environments, a pod can only expose a predetermined set of
ports. Of course the administrator can customize this set, but in some
cases (e.g. openshift.io) it is very unlikely that the extra port is added.


>
>
>>
>> 2017-06-30 11:40 GMT+02:00 Thomas SEGISMONT <tsegismont at gmail.com>:
>>
>>> Hi everyone,
>>>
>>> Thank you for this great work, the dependency diet and the extra port
>>> removal are both very useful. The extra port removal is key to enable
>>> Vert.x clustering in Openshift S2I environments.
>>>
>>> I tried the new KUBE_PING (beta1) with vertx-infinispan and it worked
>>> fine. I have a few questions though.
>>>
>>> I couldn't configure it with env variables. Before you ask, yes I
>>> noticed the name changes ;-) I only had a quick look at JGroups config code
>>> but it seems it only resolves system properties. Did it work for you
>>> because you tried with an Infinispan server?
>>>
>>
> Could you please tell me what is the JGroups version you're running on? I
> think environment variables support requires >= 4.0.3.Final [1].
>
> [1] https://github.com/jgroups-extras/jgroups-kubernetes/blob/master/pom.
> xml#L56
>

This explains it. I use the version of JGroups which comes with ISPN
9.0.0.Final (4.0.1.Final)


>
>
>>
>>> Since I couldn't configure it externally I had to create a custom
>>> JGroups file. Usually, we recommend [1] Vert.x users to add the
>>> infinispan-cloud dependency and a system property:
>>> -Dvertx.jgroups.config=default-configs/default-jgroups-kubernetes.xml
>>>
>>
> +1, that's the correct way to do it. I think bumping up JGroups version
> might solve your problem here.
>

Yes. Upgrading JGroups that solved the issue.


>
>
>>
>>> My custom JGroups file is a just a copy of default-configs/default-jgroups-kubernetes.xml
>>> in which I added the masterHost and namespace properties.
>>>
>>
> hmmmm that's odd. Why do you need to change masterHost? The default should
> be fine in 99% of the use cases. The namespace is a separate thing and in
> most of the cases a user should set it to his own project.
>

I needed to set the masterHost via a sysprop as my older version of JGroups
wouldn't lookup env vars. With 4.0.3.Final I don't need it anymore.


>
>
>> Is it still recommended to use the default-configs/default-jgroups-kubernetes.xml
>>> stack ? Or is any change planned after the KUBE_PING changes?
>>> I wouldn't expect a protocol implementation change to impact a stack
>>> configuration but they say there are no stupid questions :)
>>>
>>
> No no, using default-jgroups-kubernetes.xml is still necessary (and there
> are no plans to change it in the future). Using specific transport is tied
> with your deployment model. In most of the cases in Kubernetes and
> OpenShift you should use KUBE_PING. Your network configuration might
> support multicasting and then you'd probably need to check if UDP is not
> performing better. You may also use StatefulSets and try out DNS_PING.
>
> As you can see there are many different combinations of protocols you
> might use. Recommending default config shipped with infinispan-cloud is a
> way to go here in my opinion.
>

OK. I had no plans to try my own stack for Openshift really :) Just wanted
to make sure the new KUBE_PING doesn't impact the infinispan-cloud default
Kubernetes stack.


>
>
>>
>>> Thank you,
>>> Thomas
>>>
>>>
>>> [1] http://vertx.io/docs/vertx-infinispan/java/#_
>>> configuring_for_openshift_3
>>>
>>>
>>> 2017-06-15 8:21 GMT+02:00 Sebastian Laskawiec <slaskawi at redhat.com>:
>>>
>>>> Yep, no problems found!!!
>>>>
>>>> I had also impression that the new implementation is "faster". Though I
>>>> haven't measured it... it just my impression.
>>>>
>>>> Awesome work Bela!
>>>>
>>>> On Thu, Jun 15, 2017 at 7:42 AM Bela Ban <belaban at mailbox.org> wrote:
>>>>
>>>>> Thanks, Sebastian!
>>>>>
>>>>> I assume testing on GKE and minikube/openshift was successful?
>>>>>
>>>>>
>>>>> On 14/06/17 13:15, Sebastian Laskawiec wrote:
>>>>> > Hey guys,
>>>>> >
>>>>> > Just a heads up, I've just created a PR that upgrades KUBE_PING to
>>>>> > 1.0.0.Beta1 [1]. As you probably seen in [2], 1.0.0.Beta1 was
>>>>> completely
>>>>> > rewritten and might behave slightly differently.
>>>>> >
>>>>> > Here is a summary of changes:
>>>>> >
>>>>> >   * The latest KUBE_PING doesn't require embedded HTTP server for
>>>>> >     discovery. Thus it is no longer required to expose port 8888 in
>>>>> Pods.
>>>>> >   * The number of dependencies has been decreased. Currently we only
>>>>> >     require JGroups and OAuth library.
>>>>> >   * The new KUBE_PING works only with JGroups 4. There will be no
>>>>> >     JGroups 3 support.
>>>>> >   * Some of the environmental variables were shortened and we removed
>>>>> >     `OPENSHIFT` prefix. So if you use `OPENSHIFT_KUBE_PING_
>>>>> NAMESPACE`,
>>>>> >     you will need to change it to `KUBERNETES_NAMESPACE`. Please
>>>>> refer
>>>>> >     to [3] for more information.
>>>>> >
>>>>> > I also switched default branch in Kubernetes Ping repository to
>>>>> master [4].
>>>>> >
>>>>> > Thanks,
>>>>> > Sebastian
>>>>> >
>>>>> > [1] https://github.com/infinispan/infinispan/pull/5201
>>>>> > [2] http://belaban.blogspot.ch/2017/05/running-infinispan-
>>>>> cluster-with.html
>>>>> > [3] https://github.com/jgroups-extras/jgroups-kubernetes/
>>>>> blob/master/README.adoc
>>>>> > [4] https://github.com/jgroups-extras/jgroups-kubernetes
>>>>> > --
>>>>> >
>>>>> > SEBASTIAN ŁASKAWIEC
>>>>> >
>>>>> > INFINISPAN DEVELOPER
>>>>> >
>>>>> > Red Hat EMEA <https://www.redhat.com/>
>>>>> >
>>>>> > <https://red.ht/sig>
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > infinispan-dev mailing list
>>>>> > infinispan-dev at lists.jboss.org
>>>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> >
>>>>>
>>>>> --
>>>>> Bela Ban | http://www.jgroups.org
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>>
>>>> SEBASTIAN ŁASKAWIEC
>>>>
>>>> INFINISPAN DEVELOPER
>>>>
>>>> Red Hat EMEA <https://www.redhat.com/>
>>>> <https://red.ht/sig>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
>
> SEBASTIAN ŁASKAWIEC
>
> INFINISPAN DEVELOPER
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170703/0c19a4d8/attachment-0001.html

More information about the infinispan-dev mailing list