Re: [infinispan-dev] [wildfly-dev] WFLYTX0013 in the Infinispan Openshift Template

Thanks for confirming Brian. Perhaps we could set it to: node-identifier="${jboss.tx.node.id:1}" (a bit like https://github.com/jboss-developer/jboss-eap- quickstarts/tree/7.1/jts) Sebastian could set -Djboss.tx.node.id during startup in a script? On 7 May 2018 at 22:08, Brian Stansberry <brian.stansberry(a)redhat.com&gt; wrote: > If it's not already set, WildFly sets the system property jboss.node.name > at the very beginning of server boot, so ${jboss.node.name*:1*} would > not resolve to 1. > > On Sun, May 6, 2018 at 6:10 PM, Sebastian Laskawiec <slaskawi(a)redhat.com&gt; > wrote: > >> Ok, so how about doing the same thing you suggested, but just more >> explicitly - adding node-identifier="${jboss.node.name*:1*}". This way >> the bare metal deployment should be happy (since the default is still 1) >> and we wouldn't need to override it in Infinispan. >> >> On Tue, May 1, 2018 at 10:09 AM Tom Jenkinson <tom.jenkinson(a)redhat.com&gt; >> wrote: >> >>> I am not sure - the default should be "1" for the bare metal case so >>> the warning is reliably triggered but the default can be the pod name for >>> OpenShift templates that only allow a single instance of the application >>> server - does that help? >>> >>> The file you looked to want to edit is shared by bare metal and other >>> deployment environments so it would be confusing to set the default to >>> jboss.node.name there IMO. >>> >>> On 1 May 2018 at 03:39, Sebastian Laskawiec <slaskawi(a)redhat.com&gt; >>> wrote: >>> >>>> Fair enough Tom. Thanks for explanation. >>>> >>>> One more request - would you guys be OK with me adding >>>> a node-identifier="${jboss.node.name}" to the transaction subsystem >>>> template [1]? This way we wouldn't need to copy it into Infinispan (since >>>> we need to set it). >>>> >>>> [1] https://github.com/wildfly/wildfly/blob/master/transacti >>>> ons/src/main/resources/subsystem-templates/transactions.xml#L6 >>>> >>>> On Wed, Apr 18, 2018 at 3:37 PM Tom Jenkinson < >>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>> >>>>> On 18 April 2018 at 14:07, Sebastian Laskawiec <slaskawi(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> Hey Tom, >>>>>> >>>>>> Comments inlined. >>>>>> >>>>>> Thanks, >>>>>> Sebastian >>>>>> >>>>>> On Tue, Apr 17, 2018 at 4:37 PM Tom Jenkinson < >>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On 16 April 2018 at 09:31, <> wrote: >>>>>>> >>>>>>>> Adding +WildFly Dev <wildfly-dev at lists.jboss.org> to the loop >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks for the explanation Rado. >>>>>>>> >>>>>>>> TL;DR: A while ago Sanne pointed out that we do not set >>>>>>>> `node-identifier` >>>>>>>> in transaction subsystem by default. The default value for the >>>>>>>> `node-identifier` attribute it `1`. Not setting this attribute >>>>>>>> might cause >>>>>>>> problems in transaction recovery. Perhaps we could follow Rado's >>>>>>>> idea and >>>>>>>> set it to node name by default? >>>>>>>> >>>>>>> Indeed - it would cause serious data integrity problems if a >>>>>>> non-unique node-identifier is used. >>>>>>> >>>>>>>> Some more comments inlined. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Sebastian >>>>>>>> >>>>>>>> On Fri, Apr 13, 2018 at 7:07 PM Radoslav Husar <rhusar at >>>>>>>> redhat.com> wrote: >>>>>>>> >>>>>>>> > Hi Sebastian, >>>>>>>> > >>>>>>>> > On Wed, Apr 11, 2018 at 2:31 PM, Sebastian Laskawiec >>>>>>>> > <slaskawi at redhat.com> wrote: >>>>>>>> > > Hey Rado, Paul, >>>>>>>> > > >>>>>>>> > > I started looking into this issue and it turned out that WF >>>>>>>> subsystem >>>>>>>> > > template doesn't provide `node-identifier` attribute [1]. >>>>>>>> > >>>>>>>> > I assume you mean that the default WildFly server profiles do not >>>>>>>> >>>>>>> > explicitly define the attribute. Right ? thus the value defaults >>>>>>>> in >>>>>>> >>>>>>> >>>>>>>> > the model to "1" >>>>>>>> > >>>>>>>> > https://github.com/wildfly/wildfly/blob/master/transactions/ >>>>>>>> src/main/java/org/jboss/as/txn/subsystem/TransactionSubsyste >>>>>>>> mRootResourceDefinition.java#L97 >>>>>>>> > which sole intention seems to be to log a warning on boot if the >>>>>>>> value >>>>>>>> > is unchanged. >>>>>>>> > Why they decided on a constant that will be inherently not >>>>>>>> unique as >>>>>>>> > opposed to defaulting to the node name (which we already require >>>>>>>> to be >>>>>>>> > unique) as clustering node name or undertow instance-id does, is >>>>>>>> > unclear to me. >>>>>>>> > Some context is on https://issues.jboss.org/browse/WFLY-1119. >>>>>>>> > >>>>>>>> >>>>>>>> In OpenShift environment we could set it to `hostname`. This is >>>>>>>> guaranteed >>>>>>>> to be unique in whole OpenShift cluster. >>>>>>>> >>>>>>>> >>>>>>>> We do this too in EAP images. >>>>>>> To Rado's point, the default is "1" so we can print the warning to >>>>>>> alert people they are misconfigured - it seems to be working :) >>>>>>> >>>>>> >>>>>> This is the point. From my understanding, if we set it to node name >>>>>> (instead of "1"), we could make it always work correctly. We could even >>>>>> remove the code that emits the warning (since the node name needs to be >>>>>> unique). >>>>>> >>>>>> To sum it up - if we decided to proceed this way, there would be no >>>>>> requirement of setting the node-identifier at all. >>>>>> >>>>> >>>>> For OpenShift you are right there is no requirement for someone to >>>>> change the node-identifier from the podname and so that is why EAP images >>>>> do that. >>>>> >>>>> For bare-metal it is different as there can be two servers on the >>>>> same machine so they were configured to use the hostname as they >>>>> node-identifier then if they were also connected to the same resource >>>>> managers or the same object store they would interfere with each other. >>>>> >>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>>> > >>>>>>> >>>>>>> >>>>>>>> > > I'm not sure if you guys are the right people to ask, but is >>>>>>>> it safe to >>>>>>>> > > leave it set to default? Or shall I override our Infinispan >>>>>>>> templates and >>>>>>>> > > add this parameter (as I mentioned before, in OpenShift this I >>>>>>>> wanted to >>>>>>>> > set >>>>>>>> > > it as Pod name trimmed to the last 23 chars since this is the >>>>>>>> limit). >>>>>>>> >>>>>>> Putting a response to this in line - I am not certain who >>>>>>> originally proposed this. >>>>>>> >>>>>>> You must use a globally unique node-identifier. If you are certain >>>>>>> the last 23 characters guarantee that it would be valid - if there is a >>>>>>> chance they are not unique it is not valid to trim. >>>>>>> >>>>>> >>>>>> If that's not an issue, again, we could use the same limit as we >>>>>> have for node name. >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> > >>>>>>> >>>>>>> >>>>>>>> > It is not safe to leave it set to "1" as that results in >>>>>>>> inconsistent >>>>>>>> > processing of transaction recovery. >>>>>>>> > IIUC we already set it to the node name for both EAP and JDG >>>>>>>> > >>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/master/os >>>>>>>> -eap70-openshift/added/standalone-openshift.xml#L411 >>>>>>>> > >>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/master/os >>>>>>>> -jdg7-conffiles/added/clustered-openshift.xml#L282 >>>>>>>> >>>>>>> > which in turn defaults to the pod name ? so which profiles are we >>>>>>> >>>>>>> >>>>>>>> > talking about here? >>>>>>>> > >>>>>>>> >>>>>>>> Granted, we set it by default in CCT Modules. However in >>>>>>>> Infinispan we just >>>>>>>> grab provided transaction subsystem when rendering full >>>>>>>> configuration from >>>>>>>> featurepacks: >>>>>>>> https://github.com/infinispan/infinispan/blob/master/server/ >>>>>>>> integration/feature-pack/src/main/resources/configuration/st >>>>>>>> andalone/subsystems-cloud.xml#L19 >>>>>>>> >>>>>>>> The default configuration XML doesn't contain the `node-identifier` >>>>>>>> attribute. I can add it manually in the cloud.xml but I believe >>>>>>>> the right >>>>>>>> approach is to modify the transaction subsystem. >>>>>>>> >>>>>>>> >>>>>>>> > Rado >>>>>>>> > >>>>>>>> > > Thanks, >>>>>>>> > > Seb >>>>>>>> > > >>>>>>>> > > [1] usually set to node-identifier="${jboss.node.name}" >>>>>>>> > > >>>>>>>> > > >>>>>>>> >>>>>>> > > On Mon, Apr 9, 2018 at 10:39 AM Sanne Grinovero <sanne at >>>>>>>> infinispan.org> >>>>>>>> > > wrote: >>>>>>>> > >> >>>>>>>> > >> On 9 April 2018 at 09:26, Sebastian Laskawiec <slaskawi at >>>>>>>> redhat.com> >>>>>>> >>>>>>> >>>>>>>> > wrote: >>>>>>>> > >> > Thanks for looking into it Sanne. Of course, we should add >>>>>>>> it (it can >>>>>>>> > be >>>>>>>> > >> > set >>>>>>>> > >> > to the same name as hostname since those are unique in >>>>>>>> Kubernetes). >>>>>>>> > >> > >>>>>>>> > >> > Created https://issues.jboss.org/browse/ISPN-9051 for it. >>>>>>>> > >> > >>>>>>>> > >> > Thanks again! >>>>>>>> > >> > Seb >>>>>>>> > >> >>>>>>>> > >> Thanks Sebastian! >>>>>>>> > >> >>>>>>>> > >> > >>>>>>>> >>>>>>> > >> > On Fri, Apr 6, 2018 at 8:53 PM Sanne Grinovero <sanne at >>>>>>>> infinispan.org> >>>>>>> >>>>>>> >>>>>>>> > >> > wrote: >>>>>>>> > >> >> >>>>>>>> > >> >> Hi all, >>>>>>>> > >> >> >>>>>>>> > >> >> I've started to use the Infinispan Openshift Template and >>>>>>>> was >>>>>>>> > browsing >>>>>>>> > >> >> through the errors and warnings this produces. >>>>>>>> > >> >> >>>>>>>> > >> >> In particular I noticed "WFLYTX0013: Node identifier >>>>>>>> property is set >>>>>>>> > >> >> to the default value. Please make sure it is unique." >>>>>>>> being produced >>>>>>>> > >> >> by the transaction system. >>>>>>>> > >> >> >>>>>>>> > >> >> The node id is usually not needed for developer's >>>>>>>> convenience and >>>>>>>> > >> >> assuming there's a single node in "dev mode", yet clearly >>>>>>>> the >>>>>>>> > >> >> Infinispan template is meant to work with multiple nodes >>>>>>>> running so >>>>>>>> > >> >> this warning seems concerning. >>>>>>>> > >> >> >>>>>>>> > >> >> I'm not sure what the impact is on the transaction manager >>>>>>>> so I asked >>>>>>>> > >> >> on the Narayana forums; Tom pointed me to some thourough >>>>>>>> design >>>>>>>> > >> >> documents and also suggested the EAP image does set the >>>>>>>> node >>>>>>>> > >> >> identifier: >>>>>>>> > >> >> - https://developer.jboss.org/message/981702#981702 >>>>>>>> > >> >> >>>>>>>> > >> >> WDYT? we probably want the Infinispan template to set this >>>>>>>> as well, >>>>>>>> > or >>>>>>>> > >> >> silence the warning? >>>>>>>> > >> >> >>>>>>>> > >> >> Thanks, >>>>>>>> > >> >> Sanne >>>>>>>> > >> >> _______________________________________________ >>>>>>>> > >> >> infinispan-dev mailing list >>>>>>>> >>>>>>> > >> >> infinispan-dev at lists.jboss.org >>>>>>> >>>>>>> >>>>>>>> > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>> > >> > >>>>>>>> > >> > >>>>>>>> > >> > _______________________________________________ >>>>>>>> > >> > infinispan-dev mailing list >>>>>>>> >>>>>>> > >> > infinispan-dev at lists.jboss.org >>>>>>> >>>>>>> >>>>>>>> > >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>> > >> _______________________________________________ >>>>>>>> > >> infinispan-dev mailing list >>>>>>>> > >> infinispan-dev at lists.jboss.org >>>>>>>> > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>> > >>>>>>>> -------------- next part -------------- >>>>>>>> An HTML attachment was scrubbed... >>>>>>>> URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/201 >>>>>>>> 80416/65962cf1/attachment-0001.html >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>> >> _______________________________________________ >> wildfly-dev mailing list >> wildfly-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/wildfly-dev >> > > > > -- > Brian Stansberry > Manager, Senior Principal Software Engineer > Red Hat >