Re: [wildfly-dev] [infinispan-dev] WFLYTX0013 in the Infinispan Openshift Template

Hi Sebastian, I am not sure if you raised an issue for this but I have just found one from Brad that was similar so I am using that one: https://issues.jboss.org/browse/WFLY-10541 Thanks, Tom On 10 July 2018 at 14:32, Sebastian Laskawiec <slaskawi(a)redhat.com&gt; wrote: > Hey Tom! > > Could you please tell me what is the status of this? It seems one of the > Keycloak users got hit by this problem: http://lists.jboss.or > g/pipermail/keycloak-dev/2018-July/010985.html > > Thanks, > Sebastian > > On Thu, May 10, 2018 at 11:10 AM Tom Jenkinson <tom.jenkinson(a)redhat.com&gt; > wrote: > >> Sure - thanks! >> >> On 10 May 2018 at 02:56, Sebastian Laskawiec <slaskawi(a)redhat.com&gt; >> wrote: >> >>> I'm sorry for delay... I got sucked by the Summit prep activities. >>> >>> Yes, to all, what you said! Shall I create a JIRA for you? >>> >>> >>> >>> On Wed, May 9, 2018 at 9:39 AM Tom Jenkinson <tom.jenkinson(a)redhat.com&gt; >>> wrote: >>> >>>> Thanks Brian. Does it work for you Sebastian? >>>> >>>> On 8 May 2018 at 23:05, Brian Stansberry <brian.stansberry(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Ah, ok. I was thinking of scripting in the broad sense the various >>>>> stuff that goes into creating images. >>>>> >>>>> In any case, I don't see any downside to having node-identifier="${ >>>>> jboss.tx.node.id:1}" in the standard WF config files. >>>>> >>>>> >>>>> >>>>> On Tue, May 8, 2018 at 3:07 PM, Tom Jenkinson < >>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>> >>>>>> I think they want to avoid changing the standalone.xml file and just >>>>>> want to control it from their startup script. >>>>>> >>>>>> On 8 May 2018 at 18:45, Brian Stansberry < >>>>>> brian.stansberry(a)redhat.com&gt; wrote: >>>>>> >>>>>>> I might have missed something along the way, but if they are going >>>>>>> to do scripting wouldn't they just set the attribute to ${ >>>>>>> jboss.node.name} and count on the fact that this is unique per pod? >>>>>>> >>>>>>> On Tue, May 8, 2018 at 3:28 AM, Tom Jenkinson < >>>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> Thanks for confirming Brian. >>>>>>>> >>>>>>>> Perhaps we could set it to: >>>>>>>> node-identifier="${jboss.tx.node.id:1}" >>>>>>>> (a bit like https://github.com/jboss- >>>>>>>> developer/jboss-eap-quickstarts/tree/7.1/jts) >>>>>>>> >>>>>>>> Sebastian could set -Djboss.tx.node.id during startup in a script? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 7 May 2018 at 22:08, Brian Stansberry < >>>>>>>> brian.stansberry(a)redhat.com&gt; wrote: >>>>>>>> >>>>>>>>> If it's not already set, WildFly sets the system property >>>>>>>>> jboss.node.name at the very beginning of server boot, so ${ >>>>>>>>> jboss.node.name*:1*} would not resolve to 1. >>>>>>>>> >>>>>>>>> On Sun, May 6, 2018 at 6:10 PM, Sebastian Laskawiec < >>>>>>>>> slaskawi(a)redhat.com&gt; wrote: >>>>>>>>> >>>>>>>>>> Ok, so how about doing the same thing you suggested, but just >>>>>>>>>> more explicitly - adding node-identifier="${jboss.node.name*:1*}". >>>>>>>>>> This way the bare metal deployment should be happy (since the default is >>>>>>>>>> still 1) and we wouldn't need to override it in Infinispan. >>>>>>>>>> >>>>>>>>>> On Tue, May 1, 2018 at 10:09 AM Tom Jenkinson < >>>>>>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>>>>>> >>>>>>>>>>> I am not sure - the default should be "1" for the bare metal >>>>>>>>>>> case so the warning is reliably triggered but the default can be the pod >>>>>>>>>>> name for OpenShift templates that only allow a single instance of the >>>>>>>>>>> application server - does that help? >>>>>>>>>>> >>>>>>>>>>> The file you looked to want to edit is shared by bare metal and >>>>>>>>>>> other deployment environments so it would be confusing to set the default >>>>>>>>>>> to jboss.node.name there IMO. >>>>>>>>>>> >>>>>>>>>>> On 1 May 2018 at 03:39, Sebastian Laskawiec < >>>>>>>>>>> slaskawi(a)redhat.com&gt; wrote: >>>>>>>>>>> >>>>>>>>>>>> Fair enough Tom. Thanks for explanation. >>>>>>>>>>>> >>>>>>>>>>>> One more request - would you guys be OK with me adding >>>>>>>>>>>> a node-identifier="${jboss.node.name}" to the transaction >>>>>>>>>>>> subsystem template [1]? This way we wouldn't need to copy it into >>>>>>>>>>>> Infinispan (since we need to set it). >>>>>>>>>>>> >>>>>>>>>>>> [1] https://github.com/wildfly/wildfly/blob/master/transacti >>>>>>>>>>>> ons/src/main/resources/subsystem-templates/transactions.xml#L6 >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 18, 2018 at 3:37 PM Tom Jenkinson < >>>>>>>>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On 18 April 2018 at 14:07, Sebastian Laskawiec < >>>>>>>>>>>>> slaskawi(a)redhat.com&gt; wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hey Tom, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Comments inlined. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Sebastian >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Apr 17, 2018 at 4:37 PM Tom Jenkinson < >>>>>>>>>>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 16 April 2018 at 09:31, <> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Adding +WildFly Dev <wildfly-dev at lists.jboss.org> to >>>>>>>>>>>>>>>> the loop >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for the explanation Rado. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> TL;DR: A while ago Sanne pointed out that we do not set >>>>>>>>>>>>>>>> `node-identifier` >>>>>>>>>>>>>>>> in transaction subsystem by default. The default value for >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> `node-identifier` attribute it `1`. Not setting this >>>>>>>>>>>>>>>> attribute might cause >>>>>>>>>>>>>>>> problems in transaction recovery. Perhaps we could follow >>>>>>>>>>>>>>>> Rado's idea and >>>>>>>>>>>>>>>> set it to node name by default? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Indeed - it would cause serious data integrity problems if >>>>>>>>>>>>>>> a non-unique node-identifier is used. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Some more comments inlined. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Sebastian >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, Apr 13, 2018 at 7:07 PM Radoslav Husar <rhusar at >>>>>>>>>>>>>>>> redhat.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > Hi Sebastian, >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > On Wed, Apr 11, 2018 at 2:31 PM, Sebastian Laskawiec >>>>>>>>>>>>>>>> > <slaskawi at redhat.com> wrote: >>>>>>>>>>>>>>>> > > Hey Rado, Paul, >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > I started looking into this issue and it turned out >>>>>>>>>>>>>>>> that WF subsystem >>>>>>>>>>>>>>>> > > template doesn't provide `node-identifier` attribute >>>>>>>>>>>>>>>> [1]. >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > I assume you mean that the default WildFly server >>>>>>>>>>>>>>>> profiles do not >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > explicitly define the attribute. Right ? thus the value >>>>>>>>>>>>>>>> defaults in >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > the model to "1" >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > https://github.com/wildfly/wil >>>>>>>>>>>>>>>> dfly/blob/master/transactions/src/main/java/org/jboss/as/ >>>>>>>>>>>>>>>> txn/subsystem/TransactionSubsystemRootResourceDefinition. >>>>>>>>>>>>>>>> java#L97 >>>>>>>>>>>>>>>> > which sole intention seems to be to log a warning on >>>>>>>>>>>>>>>> boot if the value >>>>>>>>>>>>>>>> > is unchanged. >>>>>>>>>>>>>>>> > Why they decided on a constant that will be inherently >>>>>>>>>>>>>>>> not unique as >>>>>>>>>>>>>>>> > opposed to defaulting to the node name (which we already >>>>>>>>>>>>>>>> require to be >>>>>>>>>>>>>>>> > unique) as clustering node name or undertow instance-id >>>>>>>>>>>>>>>> does, is >>>>>>>>>>>>>>>> > unclear to me. >>>>>>>>>>>>>>>> > Some context is on https://issues.jboss.org/brows >>>>>>>>>>>>>>>> e/WFLY-1119. >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In OpenShift environment we could set it to `hostname`. >>>>>>>>>>>>>>>> This is guaranteed >>>>>>>>>>>>>>>> to be unique in whole OpenShift cluster. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We do this too in EAP images. >>>>>>>>>>>>>>> To Rado's point, the default is "1" so we can print the >>>>>>>>>>>>>>> warning to alert people they are misconfigured - it seems to be working :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> This is the point. From my understanding, if we set it to >>>>>>>>>>>>>> node name (instead of "1"), we could make it always work correctly. We >>>>>>>>>>>>>> could even remove the code that emits the warning (since the node name >>>>>>>>>>>>>> needs to be unique). >>>>>>>>>>>>>> >>>>>>>>>>>>>> To sum it up - if we decided to proceed this way, there >>>>>>>>>>>>>> would be no requirement of setting the node-identifier at all. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> For OpenShift you are right there is no requirement for >>>>>>>>>>>>> someone to change the node-identifier from the podname and so that is why >>>>>>>>>>>>> EAP images do that. >>>>>>>>>>>>> >>>>>>>>>>>>> For bare-metal it is different as there can be two servers on >>>>>>>>>>>>> the same machine so they were configured to use the hostname as they >>>>>>>>>>>>> node-identifier then if they were also connected to the same resource >>>>>>>>>>>>> managers or the same object store they would interfere with each other. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > > I'm not sure if you guys are the right people to ask, >>>>>>>>>>>>>>>> but is it safe to >>>>>>>>>>>>>>>> > > leave it set to default? Or shall I override our >>>>>>>>>>>>>>>> Infinispan templates and >>>>>>>>>>>>>>>> > > add this parameter (as I mentioned before, in >>>>>>>>>>>>>>>> OpenShift this I wanted to >>>>>>>>>>>>>>>> > set >>>>>>>>>>>>>>>> > > it as Pod name trimmed to the last 23 chars since this >>>>>>>>>>>>>>>> is the limit). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Putting a response to this in line - I am not certain who >>>>>>>>>>>>>>> originally proposed this. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> You must use a globally unique node-identifier. If you are >>>>>>>>>>>>>>> certain the last 23 characters guarantee that it would be valid - if there >>>>>>>>>>>>>>> is a chance they are not unique it is not valid to trim. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> If that's not an issue, again, we could use the same limit >>>>>>>>>>>>>> as we have for node name. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > It is not safe to leave it set to "1" as that results in >>>>>>>>>>>>>>>> inconsistent >>>>>>>>>>>>>>>> > processing of transaction recovery. >>>>>>>>>>>>>>>> > IIUC we already set it to the node name for both EAP and >>>>>>>>>>>>>>>> JDG >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > https://github.com/jboss-opens >>>>>>>>>>>>>>>> hift/cct_module/blob/master/os-eap70-openshift/added/ >>>>>>>>>>>>>>>> standalone-openshift.xml#L411 >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > https://github.com/jboss-opens >>>>>>>>>>>>>>>> hift/cct_module/blob/master/os-jdg7-conffiles/added/ >>>>>>>>>>>>>>>> clustered-openshift.xml#L282 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > which in turn defaults to the pod name ? so which >>>>>>>>>>>>>>>> profiles are we >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > talking about here? >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Granted, we set it by default in CCT Modules. However in >>>>>>>>>>>>>>>> Infinispan we just >>>>>>>>>>>>>>>> grab provided transaction subsystem when rendering full >>>>>>>>>>>>>>>> configuration from >>>>>>>>>>>>>>>> featurepacks: >>>>>>>>>>>>>>>> https://github.com/infinispan/ >>>>>>>>>>>>>>>> infinispan/blob/master/server/ >>>>>>>>>>>>>>>> integration/feature-pack/src/m >>>>>>>>>>>>>>>> ain/resources/configuration/st >>>>>>>>>>>>>>>> andalone/subsystems-cloud.xml#L19 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The default configuration XML doesn't contain the >>>>>>>>>>>>>>>> `node-identifier` >>>>>>>>>>>>>>>> attribute. I can add it manually in the cloud.xml but I >>>>>>>>>>>>>>>> believe the right >>>>>>>>>>>>>>>> approach is to modify the transaction subsystem. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > Rado >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > > Thanks, >>>>>>>>>>>>>>>> > > Seb >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > [1] usually set to node-identifier="${jboss.node.name >>>>>>>>>>>>>>>> }" >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > > On Mon, Apr 9, 2018 at 10:39 AM Sanne Grinovero <sanne >>>>>>>>>>>>>>>> at infinispan.org> >>>>>>>>>>>>>>>> > > wrote: >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> > >> On 9 April 2018 at 09:26, Sebastian Laskawiec >>>>>>>>>>>>>>>> <slaskawi at redhat.com> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>>>> > >> > Thanks for looking into it Sanne. Of course, we >>>>>>>>>>>>>>>> should add it (it can >>>>>>>>>>>>>>>> > be >>>>>>>>>>>>>>>> > >> > set >>>>>>>>>>>>>>>> > >> > to the same name as hostname since those are unique >>>>>>>>>>>>>>>> in Kubernetes). >>>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>>> > >> > Created https://issues.jboss.org/browse/ISPN-9051 >>>>>>>>>>>>>>>> for it. >>>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>>> > >> > Thanks again! >>>>>>>>>>>>>>>> > >> > Seb >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> > >> Thanks Sebastian! >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >> > On Fri, Apr 6, 2018 at 8:53 PM Sanne Grinovero >>>>>>>>>>>>>>>> <sanne at infinispan.org> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > >> > wrote: >>>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>>> > >> >> Hi all, >>>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>>> > >> >> I've started to use the Infinispan Openshift >>>>>>>>>>>>>>>> Template and was >>>>>>>>>>>>>>>> > browsing >>>>>>>>>>>>>>>> > >> >> through the errors and warnings this produces. >>>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>>> > >> >> In particular I noticed "WFLYTX0013: Node >>>>>>>>>>>>>>>> identifier property is set >>>>>>>>>>>>>>>> > >> >> to the default value. Please make sure it is >>>>>>>>>>>>>>>> unique." being produced >>>>>>>>>>>>>>>> > >> >> by the transaction system. >>>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>>> > >> >> The node id is usually not needed for developer's >>>>>>>>>>>>>>>> convenience and >>>>>>>>>>>>>>>> > >> >> assuming there's a single node in "dev mode", yet >>>>>>>>>>>>>>>> clearly the >>>>>>>>>>>>>>>> > >> >> Infinispan template is meant to work with multiple >>>>>>>>>>>>>>>> nodes running so >>>>>>>>>>>>>>>> > >> >> this warning seems concerning. >>>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>>> > >> >> I'm not sure what the impact is on the transaction >>>>>>>>>>>>>>>> manager so I asked >>>>>>>>>>>>>>>> > >> >> on the Narayana forums; Tom pointed me to some >>>>>>>>>>>>>>>> thourough design >>>>>>>>>>>>>>>> > >> >> documents and also suggested the EAP image does >>>>>>>>>>>>>>>> set the node >>>>>>>>>>>>>>>> > >> >> identifier: >>>>>>>>>>>>>>>> > >> >> - https://developer.jboss.org/me >>>>>>>>>>>>>>>> ssage/981702#981702 >>>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>>> > >> >> WDYT? we probably want the Infinispan template to >>>>>>>>>>>>>>>> set this as well, >>>>>>>>>>>>>>>> > or >>>>>>>>>>>>>>>> > >> >> silence the warning? >>>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>>> > >> >> Thanks, >>>>>>>>>>>>>>>> > >> >> Sanne >>>>>>>>>>>>>>>> > >> >> _______________________________________________ >>>>>>>>>>>>>>>> > >> >> infinispan-dev mailing list >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >> >> infinispan-dev at lists.jboss.org >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > >> >> https://lists.jboss.org/mailma >>>>>>>>>>>>>>>> n/listinfo/infinispan-dev >>>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>>> > >> > _______________________________________________ >>>>>>>>>>>>>>>> > >> > infinispan-dev mailing list >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >> > infinispan-dev at lists.jboss.org >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > >> > https://lists.jboss.org/mailma >>>>>>>>>>>>>>>> n/listinfo/infinispan-dev >>>>>>>>>>>>>>>> > >> _______________________________________________ >>>>>>>>>>>>>>>> > >> infinispan-dev mailing list >>>>>>>>>>>>>>>> > >> infinispan-dev at lists.jboss.org >>>>>>>>>>>>>>>> > >> https://lists.jboss.org/mailma >>>>>>>>>>>>>>>> n/listinfo/infinispan-dev >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> -------------- next part -------------- >>>>>>>>>>>>>>>> An HTML attachment was scrubbed... >>>>>>>>>>>>>>>> URL: http://lists.jboss.org/piperma >>>>>>>>>>>>>>>> il/wildfly-dev/attachments/20180416/65962cf1/attachment- >>>>>>>>>>>>>>>> 0001.html >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> wildfly-dev mailing list >>>>>>>>>> wildfly-dev(a)lists.jboss.org >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Brian Stansberry >>>>>>>>> Manager, Senior Principal Software Engineer >>>>>>>>> Red Hat >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Brian Stansberry >>>>>>> Manager, Senior Principal Software Engineer >>>>>>> Red Hat >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Brian Stansberry >>>>> Manager, Senior Principal Software Engineer >>>>> Red Hat >>>>> >>>> >>>> >>