Re: [wildfly-dev] [infinispan-dev] WFLYTX0013 in the Infinispan Openshift Template

Hey Tom! Could you please tell me what is the status of this? It seems one of the Keycloak users got hit by this problem: http://lists.jboss. org/pipermail/keycloak-dev/2018-July/010985.html Thanks, Sebastian On Thu, May 10, 2018 at 11:10 AM Tom Jenkinson <tom.jenkinson(a)redhat.com&gt; wrote: > Sure - thanks! > > On 10 May 2018 at 02:56, Sebastian Laskawiec <slaskawi(a)redhat.com&gt; wrote: > >> I'm sorry for delay... I got sucked by the Summit prep activities. >> >> Yes, to all, what you said! Shall I create a JIRA for you? >> >> >> >> On Wed, May 9, 2018 at 9:39 AM Tom Jenkinson <tom.jenkinson(a)redhat.com&gt; >> wrote: >> >>> Thanks Brian. Does it work for you Sebastian? >>> >>> On 8 May 2018 at 23:05, Brian Stansberry <brian.stansberry(a)redhat.com&gt; >>> wrote: >>> >>>> Ah, ok. I was thinking of scripting in the broad sense the various >>>> stuff that goes into creating images. >>>> >>>> In any case, I don't see any downside to having node-identifier="${ >>>> jboss.tx.node.id:1}" in the standard WF config files. >>>> >>>> >>>> >>>> On Tue, May 8, 2018 at 3:07 PM, Tom Jenkinson < >>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>> >>>>> I think they want to avoid changing the standalone.xml file and just >>>>> want to control it from their startup script. >>>>> >>>>> On 8 May 2018 at 18:45, Brian Stansberry <brian.stansberry(a)redhat.com >>>>> > wrote: >>>>> >>>>>> I might have missed something along the way, but if they are going >>>>>> to do scripting wouldn't they just set the attribute to ${ >>>>>> jboss.node.name} and count on the fact that this is unique per pod? >>>>>> >>>>>> On Tue, May 8, 2018 at 3:28 AM, Tom Jenkinson < >>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>> >>>>>>> Thanks for confirming Brian. >>>>>>> >>>>>>> Perhaps we could set it to: >>>>>>> node-identifier="${jboss.tx.node.id:1}" >>>>>>> (a bit like https://github.com/jboss-developer/jboss-eap- >>>>>>> quickstarts/tree/7.1/jts) >>>>>>> >>>>>>> Sebastian could set -Djboss.tx.node.id during startup in a script? >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 7 May 2018 at 22:08, Brian Stansberry < >>>>>>> brian.stansberry(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> If it's not already set, WildFly sets the system property >>>>>>>> jboss.node.name at the very beginning of server boot, so ${ >>>>>>>> jboss.node.name*:1*} would not resolve to 1. >>>>>>>> >>>>>>>> On Sun, May 6, 2018 at 6:10 PM, Sebastian Laskawiec < >>>>>>>> slaskawi(a)redhat.com&gt; wrote: >>>>>>>> >>>>>>>>> Ok, so how about doing the same thing you suggested, but just >>>>>>>>> more explicitly - adding node-identifier="${jboss.node.name*:1*}". >>>>>>>>> This way the bare metal deployment should be happy (since the default is >>>>>>>>> still 1) and we wouldn't need to override it in Infinispan. >>>>>>>>> >>>>>>>>> On Tue, May 1, 2018 at 10:09 AM Tom Jenkinson < >>>>>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>>>>> >>>>>>>>>> I am not sure - the default should be "1" for the bare metal >>>>>>>>>> case so the warning is reliably triggered but the default can be the pod >>>>>>>>>> name for OpenShift templates that only allow a single instance of the >>>>>>>>>> application server - does that help? >>>>>>>>>> >>>>>>>>>> The file you looked to want to edit is shared by bare metal and >>>>>>>>>> other deployment environments so it would be confusing to set the default >>>>>>>>>> to jboss.node.name there IMO. >>>>>>>>>> >>>>>>>>>> On 1 May 2018 at 03:39, Sebastian Laskawiec <slaskawi(a)redhat.com >>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>>>> Fair enough Tom. Thanks for explanation. >>>>>>>>>>> >>>>>>>>>>> One more request - would you guys be OK with me adding >>>>>>>>>>> a node-identifier="${jboss.node.name}" to the transaction >>>>>>>>>>> subsystem template [1]? This way we wouldn't need to copy it into >>>>>>>>>>> Infinispan (since we need to set it). >>>>>>>>>>> >>>>>>>>>>> [1] https://github.com/wildfly/wildfly/blob/master/ >>>>>>>>>>> transactions/src/main/resources/subsystem-templates/ >>>>>>>>>>> transactions.xml#L6 >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 18, 2018 at 3:37 PM Tom Jenkinson < >>>>>>>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>>>>>>> >>>>>>>>>>>> On 18 April 2018 at 14:07, Sebastian Laskawiec < >>>>>>>>>>>> slaskawi(a)redhat.com&gt; wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey Tom, >>>>>>>>>>>>> >>>>>>>>>>>>> Comments inlined. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Sebastian >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Apr 17, 2018 at 4:37 PM Tom Jenkinson < >>>>>>>>>>>>> tom.jenkinson(a)redhat.com&gt; wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 16 April 2018 at 09:31, <> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Adding +WildFly Dev <wildfly-dev at lists.jboss.org> to >>>>>>>>>>>>>>> the loop >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for the explanation Rado. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TL;DR: A while ago Sanne pointed out that we do not set >>>>>>>>>>>>>>> `node-identifier` >>>>>>>>>>>>>>> in transaction subsystem by default. The default value for >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> `node-identifier` attribute it `1`. Not setting this >>>>>>>>>>>>>>> attribute might cause >>>>>>>>>>>>>>> problems in transaction recovery. Perhaps we could follow >>>>>>>>>>>>>>> Rado's idea and >>>>>>>>>>>>>>> set it to node name by default? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Indeed - it would cause serious data integrity problems if a >>>>>>>>>>>>>> non-unique node-identifier is used. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Some more comments inlined. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Sebastian >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Fri, Apr 13, 2018 at 7:07 PM Radoslav Husar <rhusar at >>>>>>>>>>>>>>> redhat.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > Hi Sebastian, >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > On Wed, Apr 11, 2018 at 2:31 PM, Sebastian Laskawiec >>>>>>>>>>>>>>> > <slaskawi at redhat.com> wrote: >>>>>>>>>>>>>>> > > Hey Rado, Paul, >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > I started looking into this issue and it turned out >>>>>>>>>>>>>>> that WF subsystem >>>>>>>>>>>>>>> > > template doesn't provide `node-identifier` attribute >>>>>>>>>>>>>>> [1]. >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > I assume you mean that the default WildFly server >>>>>>>>>>>>>>> profiles do not >>>>>>>>>>>>>>> >>>>>>>>>>>>>> > explicitly define the attribute. Right ? thus the value >>>>>>>>>>>>>>> defaults in >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > the model to "1" >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > https://github.com/wildfly/wildfly/blob/master/ >>>>>>>>>>>>>>> transactions/src/main/java/org/jboss/as/txn/subsystem/ >>>>>>>>>>>>>>> TransactionSubsystemRootResourceDefinition.java#L97 >>>>>>>>>>>>>>> > which sole intention seems to be to log a warning on boot >>>>>>>>>>>>>>> if the value >>>>>>>>>>>>>>> > is unchanged. >>>>>>>>>>>>>>> > Why they decided on a constant that will be inherently >>>>>>>>>>>>>>> not unique as >>>>>>>>>>>>>>> > opposed to defaulting to the node name (which we already >>>>>>>>>>>>>>> require to be >>>>>>>>>>>>>>> > unique) as clustering node name or undertow instance-id >>>>>>>>>>>>>>> does, is >>>>>>>>>>>>>>> > unclear to me. >>>>>>>>>>>>>>> > Some context is on https://issues.jboss.org/ >>>>>>>>>>>>>>> browse/WFLY-1119. >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In OpenShift environment we could set it to `hostname`. >>>>>>>>>>>>>>> This is guaranteed >>>>>>>>>>>>>>> to be unique in whole OpenShift cluster. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We do this too in EAP images. >>>>>>>>>>>>>> To Rado's point, the default is "1" so we can print the >>>>>>>>>>>>>> warning to alert people they are misconfigured - it seems to be working :) >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> This is the point. From my understanding, if we set it to >>>>>>>>>>>>> node name (instead of "1"), we could make it always work correctly. We >>>>>>>>>>>>> could even remove the code that emits the warning (since the node name >>>>>>>>>>>>> needs to be unique). >>>>>>>>>>>>> >>>>>>>>>>>>> To sum it up - if we decided to proceed this way, there would >>>>>>>>>>>>> be no requirement of setting the node-identifier at all. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> For OpenShift you are right there is no requirement for >>>>>>>>>>>> someone to change the node-identifier from the podname and so that is why >>>>>>>>>>>> EAP images do that. >>>>>>>>>>>> >>>>>>>>>>>> For bare-metal it is different as there can be two servers on >>>>>>>>>>>> the same machine so they were configured to use the hostname as they >>>>>>>>>>>> node-identifier then if they were also connected to the same resource >>>>>>>>>>>> managers or the same object store they would interfere with each other. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > > I'm not sure if you guys are the right people to ask, >>>>>>>>>>>>>>> but is it safe to >>>>>>>>>>>>>>> > > leave it set to default? Or shall I override our >>>>>>>>>>>>>>> Infinispan templates and >>>>>>>>>>>>>>> > > add this parameter (as I mentioned before, in OpenShift >>>>>>>>>>>>>>> this I wanted to >>>>>>>>>>>>>>> > set >>>>>>>>>>>>>>> > > it as Pod name trimmed to the last 23 chars since this >>>>>>>>>>>>>>> is the limit). >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Putting a response to this in line - I am not certain who >>>>>>>>>>>>>> originally proposed this. >>>>>>>>>>>>>> >>>>>>>>>>>>>> You must use a globally unique node-identifier. If you are >>>>>>>>>>>>>> certain the last 23 characters guarantee that it would be valid - if there >>>>>>>>>>>>>> is a chance they are not unique it is not valid to trim. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> If that's not an issue, again, we could use the same limit as >>>>>>>>>>>>> we have for node name. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > It is not safe to leave it set to "1" as that results in >>>>>>>>>>>>>>> inconsistent >>>>>>>>>>>>>>> > processing of transaction recovery. >>>>>>>>>>>>>>> > IIUC we already set it to the node name for both EAP and >>>>>>>>>>>>>>> JDG >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/ >>>>>>>>>>>>>>> master/os-eap70-openshift/added/standalone-openshift. >>>>>>>>>>>>>>> xml#L411 >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/ >>>>>>>>>>>>>>> master/os-jdg7-conffiles/added/clustered-openshift.xml#L282 >>>>>>>>>>>>>>> >>>>>>>>>>>>>> > which in turn defaults to the pod name ? so which profiles >>>>>>>>>>>>>>> are we >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > talking about here? >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Granted, we set it by default in CCT Modules. However in >>>>>>>>>>>>>>> Infinispan we just >>>>>>>>>>>>>>> grab provided transaction subsystem when rendering full >>>>>>>>>>>>>>> configuration from >>>>>>>>>>>>>>> featurepacks: >>>>>>>>>>>>>>> https://github.com/infinispan/ >>>>>>>>>>>>>>> infinispan/blob/master/server/integration/feature-pack/src/ >>>>>>>>>>>>>>> main/resources/configuration/standalone/subsystems-cloud. >>>>>>>>>>>>>>> xml#L19 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The default configuration XML doesn't contain the >>>>>>>>>>>>>>> `node-identifier` >>>>>>>>>>>>>>> attribute. I can add it manually in the cloud.xml but I >>>>>>>>>>>>>>> believe the right >>>>>>>>>>>>>>> approach is to modify the transaction subsystem. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > Rado >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > > Thanks, >>>>>>>>>>>>>>> > > Seb >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > [1] usually set to node-identifier="${jboss.node.name}" >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> >>>>>>>>>>>>>> > > On Mon, Apr 9, 2018 at 10:39 AM Sanne Grinovero <sanne >>>>>>>>>>>>>>> at infinispan.org> >>>>>>>>>>>>>>> > > wrote: >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> > >> On 9 April 2018 at 09:26, Sebastian Laskawiec >>>>>>>>>>>>>>> <slaskawi at redhat.com> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>>> > >> > Thanks for looking into it Sanne. Of course, we >>>>>>>>>>>>>>> should add it (it can >>>>>>>>>>>>>>> > be >>>>>>>>>>>>>>> > >> > set >>>>>>>>>>>>>>> > >> > to the same name as hostname since those are unique >>>>>>>>>>>>>>> in Kubernetes). >>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>> > >> > Created https://issues.jboss.org/browse/ISPN-9051 >>>>>>>>>>>>>>> for it. >>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>> > >> > Thanks again! >>>>>>>>>>>>>>> > >> > Seb >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> > >> Thanks Sebastian! >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>> >>>>>>>>>>>>>> > >> > On Fri, Apr 6, 2018 at 8:53 PM Sanne Grinovero <sanne >>>>>>>>>>>>>>> at infinispan.org> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >> > wrote: >>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>> > >> >> Hi all, >>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>> > >> >> I've started to use the Infinispan Openshift >>>>>>>>>>>>>>> Template and was >>>>>>>>>>>>>>> > browsing >>>>>>>>>>>>>>> > >> >> through the errors and warnings this produces. >>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>> > >> >> In particular I noticed "WFLYTX0013: Node >>>>>>>>>>>>>>> identifier property is set >>>>>>>>>>>>>>> > >> >> to the default value. Please make sure it is >>>>>>>>>>>>>>> unique." being produced >>>>>>>>>>>>>>> > >> >> by the transaction system. >>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>> > >> >> The node id is usually not needed for developer's >>>>>>>>>>>>>>> convenience and >>>>>>>>>>>>>>> > >> >> assuming there's a single node in "dev mode", yet >>>>>>>>>>>>>>> clearly the >>>>>>>>>>>>>>> > >> >> Infinispan template is meant to work with multiple >>>>>>>>>>>>>>> nodes running so >>>>>>>>>>>>>>> > >> >> this warning seems concerning. >>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>> > >> >> I'm not sure what the impact is on the transaction >>>>>>>>>>>>>>> manager so I asked >>>>>>>>>>>>>>> > >> >> on the Narayana forums; Tom pointed me to some >>>>>>>>>>>>>>> thourough design >>>>>>>>>>>>>>> > >> >> documents and also suggested the EAP image does set >>>>>>>>>>>>>>> the node >>>>>>>>>>>>>>> > >> >> identifier: >>>>>>>>>>>>>>> > >> >> - https://developer.jboss.org/ >>>>>>>>>>>>>>> message/981702#981702 >>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>> > >> >> WDYT? we probably want the Infinispan template to >>>>>>>>>>>>>>> set this as well, >>>>>>>>>>>>>>> > or >>>>>>>>>>>>>>> > >> >> silence the warning? >>>>>>>>>>>>>>> > >> >> >>>>>>>>>>>>>>> > >> >> Thanks, >>>>>>>>>>>>>>> > >> >> Sanne >>>>>>>>>>>>>>> > >> >> _______________________________________________ >>>>>>>>>>>>>>> > >> >> infinispan-dev mailing list >>>>>>>>>>>>>>> >>>>>>>>>>>>>> > >> >> infinispan-dev at lists.jboss.org >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >> >> https://lists.jboss.org/ >>>>>>>>>>>>>>> mailman/listinfo/infinispan-dev >>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>> > >> > >>>>>>>>>>>>>>> > >> > _______________________________________________ >>>>>>>>>>>>>>> > >> > infinispan-dev mailing list >>>>>>>>>>>>>>> >>>>>>>>>>>>>> > >> > infinispan-dev at lists.jboss.org >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >> > https://lists.jboss.org/mailman/listinfo/infinispan- >>>>>>>>>>>>>>> dev >>>>>>>>>>>>>>> > >> _______________________________________________ >>>>>>>>>>>>>>> > >> infinispan-dev mailing list >>>>>>>>>>>>>>> > >> infinispan-dev at lists.jboss.org >>>>>>>>>>>>>>> > >> https://lists.jboss.org/mailman/listinfo/infinispan- >>>>>>>>>>>>>>> dev >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> -------------- next part -------------- >>>>>>>>>>>>>>> An HTML attachment was scrubbed... >>>>>>>>>>>>>>> URL: http://lists.jboss.org/pipermail/wildfly-dev/ >>>>>>>>>>>>>>> attachments/20180416/65962cf1/attachment-0001.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> wildfly-dev mailing list >>>>>>>>> wildfly-dev(a)lists.jboss.org >>>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Brian Stansberry >>>>>>>> Manager, Senior Principal Software Engineer >>>>>>>> Red Hat >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Brian Stansberry >>>>>> Manager, Senior Principal Software Engineer >>>>>> Red Hat >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Brian Stansberry >>>> Manager, Senior Principal Software Engineer >>>> Red Hat >>>> >>> >>> >