<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Tue, Mar 6, 2018 at 5:11 PM Galder Zamarreño <<a href="mailto:galder@redhat.com">galder@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Sebastian Laskawiec <<a href="mailto:slaskawi@redhat.com" target="_blank">slaskawi@redhat.com</a>> writes:<br>
<br>
> Hey Galder,<br>
><br>
> Comments inlined.<br>
><br>
> Thanks,<br>
> Seb<br>
><br>
> On Fri, Mar 2, 2018 at 11:37 AM Galder Zamarreño <<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>><br>
> wrote:<br>
><br>
> Hi,<br>
><br>
> Looking at [1] and I'm wondering why the templates have to<br>
> maintain a<br>
> different XML file for OpenShift?<br>
><br>
> We already ship an XML in the server called `cloud.xml`, that<br>
> should<br>
> just work. Having a separate XML file in the templates means we're<br>
> duplicating the maintainance of XML files.<br>
><br>
> Also, users can now create caches programmatically. This is by far<br>
> the<br>
> most common tweak that had to be done to the config. So, I see the<br>
> urgency to change XML files less immediate.<br>
><br>
> So just to give you guys a bit more context - the templates were<br>
> created pretty long time ago when we didn't have admin capabilities in<br>
> Hot Rod and REST. The main argument for putting the whole<br>
> configuration into a ConfigMap was to make configuration changes<br>
> easier for the users. With ConfigMap approach they can log into<br>
> OpenShift UI, go to Resources -> ConfigMaps and edit everything using<br>
> UI. That's super convenient for hacking in my opinion. Of course, you<br>
> don't need to do that at all if you don't want. You can just spin up a<br>
> new Infinispan cluster using `oc new-app`.<br>
<br>
I agree with the usability of the ConfigMap. However, the duplication is<br>
very annoying. Would it be possible for the ConfigMap to be created on<br>
the fly out of the cloud.xml that's shipped by Infinispan Server? That<br>
way we'd still have a ConfigMap without having to duplicate XML.<br></blockquote><div><br></div><div>Probably not. This would require special permissions to call Kubernetes API from the Pod. In other words, I can't think about any other way that would work in OpenShift Online for the instance.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
> There are at least two other ways for changing the configuration that<br>
> I can think of. The first one is S2I [1][2] (long story short, you<br>
> need to put your configuration into a git repository and tell<br>
> OpenShift to build an image based on it). Even though it may seem very<br>
> convenient, it's OpenShift only solution (and there are no easy (out<br>
> of the box) options to get this running on raw Kubernetes). I'm not<br>
> judging whether it's good or bad here, just telling you how it works.<br>
> The other option would be to tell the users to do exactly the same<br>
> things we do in our templates themselves. In other words we would<br>
> remove configuration from the templates and provide a manual for the<br>
> users how to deal with configuration. I believe this is exactly what<br>
> Galder is suggesting, right?<br>
<br>
What we do in the templates right now to show users how to tweak their<br>
config is in convoluted.<br>
<br>
Ideally, adding their own custom configuration should be just a matter<br>
of:<br>
<br>
1. Creating a ConfigMap yaml pointing to an XML.<br>
2. Ask users to put their XML in a separate file pointed by the ConfigMap.<br>
3. Deploy ConfigMap and XML.<br>
4. Trigger a new Infinispan redeployment.<br></blockquote><div><br></div><div>That would probably need to be a new deployment. Most of the StatefulSet spec is immutable.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Not sure how doable this is with the current template approach, or we<br>
could explain how to do this for an already up and running application<br>
that has Infinispan created out of the default template?<br></blockquote><div><br></div><div>I've been thinking about this for a while and this is what I think we should do:</div><div><ol><li>Wait a couple of weeks and review the community image created by the CE Team. See if this is a good fit for us. If it is, I would focus on adopting this approach and adjust our templates to handle it.</li><li>Whether or not we adopt the CE community work, we could put all necessary stuff into cloud.xml or services.xml configuration. We could do one step forward and merge them together. </li><li>Make sure that dynamically created caches are persisted (this is super important!!)</li><li>Once #3 is verified we should have a decision whether or not we are adopting the CE way. At this point we could document how to use custom configuration with a ConfigMap and drop it from the templates.</li></ol><div>WDYT? Does this plan makes sense to you?</div></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
><br>
> Recently we implemented admin commands in the Hot Rod. Assuming that<br>
> caches created this way are not wiped out during restart (that needs<br>
> to be checked), we could remove the configuration from the templates<br>
> and tell the users to create their caches over Hot Rod and REST.<br>
> However we still need to have a back door for modifying configuration<br>
> manually since there are some changes that can not be done via admin<br>
> API.<br>
><br>
> [1] <a href="https://github.com/openshift/source-to-image" rel="noreferrer" target="_blank">https://github.com/openshift/source-to-image</a><br>
> [2]<br>
> <a href="https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble" rel="noreferrer" target="_blank">https://github.com/jboss-dockerfiles/infinispan/blob/master/server/.s2i/bin/assemble</a><br>
><br>
><br>
> Sure, there will always be people who modify/tweak things and<br>
> that's<br>
> fine. We should however show the people how to do that in a way<br>
> that<br>
> doesn't require us to duplicate our maintanence work.<br>
><br>
> If we think about further maintenance, I believe we should take more<br>
> things into consideration. During the last planning meeting Tristan<br>
> mentioned about bringing the project and the product closer together.<br>
> On the Cloud Enablement side of things there are ongoing experiments<br>
> to get a community images out.<br>
><br>
> If we decided to take this direction (the CE way), our templates would<br>
> need to be deprecated or will change drastically. The image will react<br>
> on different set of variables and configuration options.<br>
><br>
> Also, if we want to show the users how to use a custom XML file, I<br>
> don't<br>
> think we should show them how to embedd it in the template as JSON<br>
> [2]. It's quite a pain. Instead, the XML should be kept as a<br>
> separate<br>
> file and the JSON file reference it.<br>
><br>
> I'm still struggling to understand why this is a pain. Could you<br>
> please explain it a bit more? If you look into the maintenance guide<br>
> [3], there are only a few steps. For me it takes no longer than 15<br>
> minutes to do the upgrade. You also mentioned on IRC that this<br>
> approach is a pain for our users (I believe you mentioned something<br>
> about Ray). I also can not understand why, could you please explain it<br>
> a bit more?<br>
><br>
> [3]<br>
> <a href="https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide" rel="noreferrer" target="_blank">https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide</a><br>
><br>
><br>
> Cheers,<br>
><br>
> [1]<br>
> <a href="https://github.com/infinispan/infinispan-openshift-templates/pull/16/files" rel="noreferrer" target="_blank">https://github.com/infinispan/infinispan-openshift-templates/pull/16/files</a><br>
><br>
> [2]<br>
> <a href="https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide" rel="noreferrer" target="_blank">https://github.com/infinispan/infinispan-openshift-templates#maintenance-guide</a><br>
><br>
> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
><br>
><br>
> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</blockquote></div></div>