[wildfly-dev] [infinispan-dev] WFLYTX0013 in the Infinispan Openshift Template

Brian Stansberry brian.stansberry at redhat.com
Mon May 7 17:08:14 EDT 2018 

If it's not already set, WildFly sets the system property jboss.node.name
at the very beginning of server boot, so ${jboss.node.name*:1*} would not
resolve to 1.

On Sun, May 6, 2018 at 6:10 PM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> Ok, so how about doing the same thing you suggested, but just more
> explicitly - adding node-identifier="${jboss.node.name*:1*}". This way
> the bare metal deployment should be happy (since the default is still 1)
> and we wouldn't need to override it in Infinispan.
>
> On Tue, May 1, 2018 at 10:09 AM Tom Jenkinson <tom.jenkinson at redhat.com>
> wrote:
>
>> I am not sure - the default should be "1" for the bare metal case so the
>> warning is reliably triggered but the default can be the pod name for
>> OpenShift templates that only allow a single instance of the application
>> server - does that help?
>>
>> The file you looked to want to edit is shared by bare metal and other
>> deployment environments so it would be confusing to set the default to
>> jboss.node.name there IMO.
>>
>> On 1 May 2018 at 03:39, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>>
>>> Fair enough Tom. Thanks for explanation.
>>>
>>> One more request - would you guys be OK with me adding
>>> a node-identifier="${jboss.node.name}" to the transaction subsystem
>>> template [1]? This way we wouldn't need to copy it into Infinispan (since
>>> we need to set it).
>>>
>>> [1] https://github.com/wildfly/wildfly/blob/master/
>>> transactions/src/main/resources/subsystem-templates/transactions.xml#L6
>>>
>>> On Wed, Apr 18, 2018 at 3:37 PM Tom Jenkinson <tom.jenkinson at redhat.com>
>>> wrote:
>>>
>>>> On 18 April 2018 at 14:07, Sebastian Laskawiec <slaskawi at redhat.com>
>>>> wrote:
>>>>
>>>>> Hey Tom,
>>>>>
>>>>> Comments inlined.
>>>>>
>>>>> Thanks,
>>>>> Sebastian
>>>>>
>>>>> On Tue, Apr 17, 2018 at 4:37 PM Tom Jenkinson <
>>>>> tom.jenkinson at redhat.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 16 April 2018 at 09:31, <> wrote:
>>>>>>
>>>>>>> Adding +WildFly Dev <wildfly-dev at lists.jboss.org> to the loop
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Thanks for the explanation Rado.
>>>>>>>
>>>>>>> TL;DR: A while ago Sanne pointed out that we do not set
>>>>>>> `node-identifier`
>>>>>>> in transaction subsystem by default. The default value for the
>>>>>>> `node-identifier` attribute it `1`. Not setting this attribute might
>>>>>>> cause
>>>>>>> problems in transaction recovery. Perhaps we could follow Rado's
>>>>>>> idea and
>>>>>>> set it to node name by default?
>>>>>>>
>>>>>> Indeed - it would cause serious data integrity problems if a
>>>>>> non-unique node-identifier is used.
>>>>>>
>>>>>>> Some more comments inlined.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Sebastian
>>>>>>>
>>>>>>> On Fri, Apr 13, 2018 at 7:07 PM Radoslav Husar <rhusar at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> > Hi Sebastian,
>>>>>>> >
>>>>>>> > On Wed, Apr 11, 2018 at 2:31 PM, Sebastian Laskawiec
>>>>>>> > <slaskawi at redhat.com> wrote:
>>>>>>> > > Hey Rado, Paul,
>>>>>>> > >
>>>>>>> > > I started looking into this issue and it turned out that WF
>>>>>>> subsystem
>>>>>>> > > template doesn't provide `node-identifier` attribute [1].
>>>>>>> >
>>>>>>> > I assume you mean that the default WildFly server profiles do not
>>>>>>>
>>>>>> > explicitly define the attribute. Right ? thus the value defaults in
>>>>>>
>>>>>>
>>>>>>> > the model to "1"
>>>>>>> >
>>>>>>> > https://github.com/wildfly/wildfly/blob/master/
>>>>>>> transactions/src/main/java/org/jboss/as/txn/subsystem/
>>>>>>> TransactionSubsystemRootResourceDefinition.java#L97
>>>>>>> > which sole intention seems to be to log a warning on boot if the
>>>>>>> value
>>>>>>> > is unchanged.
>>>>>>> > Why they decided on a constant that will be inherently not unique
>>>>>>> as
>>>>>>> > opposed to defaulting to the node name (which we already require
>>>>>>> to be
>>>>>>> > unique) as clustering node name or undertow instance-id does, is
>>>>>>> > unclear to me.
>>>>>>> > Some context is on https://issues.jboss.org/browse/WFLY-1119.
>>>>>>> >
>>>>>>>
>>>>>>> In OpenShift environment we could set it to `hostname`. This is
>>>>>>> guaranteed
>>>>>>> to be unique in whole OpenShift cluster.
>>>>>>>
>>>>>>>
>>>>>>> We do this too in EAP images.
>>>>>> To Rado's point, the default is "1" so we can print the warning to
>>>>>> alert people they are misconfigured - it seems to be working :)
>>>>>>
>>>>>
>>>>> This is the point. From my understanding, if we set it to node name
>>>>> (instead of "1"), we could make it always work correctly. We could even
>>>>> remove the code that emits the warning (since the node name needs to be
>>>>> unique).
>>>>>
>>>>> To sum it up - if we decided to proceed this way, there would be no
>>>>> requirement of setting the node-identifier at all.
>>>>>
>>>>
>>>> For OpenShift you are right there is no requirement for someone to
>>>> change the node-identifier from the podname and so that is why EAP images
>>>> do that.
>>>>
>>>> For bare-metal it is different as there can be two servers on the same
>>>> machine so they were configured to use the hostname as they node-identifier
>>>> then if they were also connected to the same resource managers or the same
>>>> object store they would interfere with each other.
>>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>> >
>>>>>>
>>>>>>
>>>>>>> > > I'm not sure if you guys are the right people to ask, but is it
>>>>>>> safe to
>>>>>>> > > leave it set to default? Or shall I override our Infinispan
>>>>>>> templates and
>>>>>>> > > add this parameter (as I mentioned before, in OpenShift this I
>>>>>>> wanted to
>>>>>>> > set
>>>>>>> > > it as Pod name trimmed to the last 23 chars since this is the
>>>>>>> limit).
>>>>>>>
>>>>>> Putting a response to this in line - I am not certain who originally
>>>>>> proposed this.
>>>>>>
>>>>>> You must use a globally unique node-identifier. If you are certain
>>>>>> the last 23 characters guarantee that it would be valid - if there is a
>>>>>> chance they are not unique it is not valid to trim.
>>>>>>
>>>>>
>>>>> If that's not an issue, again, we could use the same limit as we have
>>>>> for node name.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> >
>>>>>>
>>>>>>
>>>>>>> > It is not safe to leave it set to "1" as that results in
>>>>>>> inconsistent
>>>>>>> > processing of transaction recovery.
>>>>>>> > IIUC we already set it to the node name for both EAP and JDG
>>>>>>> >
>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/
>>>>>>> master/os-eap70-openshift/added/standalone-openshift.xml#L411
>>>>>>> >
>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/
>>>>>>> master/os-jdg7-conffiles/added/clustered-openshift.xml#L282
>>>>>>>
>>>>>> > which in turn defaults to the pod name ? so which profiles are we
>>>>>>
>>>>>>
>>>>>>> > talking about here?
>>>>>>> >
>>>>>>>
>>>>>>> Granted, we set it by default in CCT Modules. However in Infinispan
>>>>>>> we just
>>>>>>> grab provided transaction subsystem when rendering full
>>>>>>> configuration from
>>>>>>> featurepacks:
>>>>>>> https://github.com/infinispan/infinispan/blob/master/server/
>>>>>>> integration/feature-pack/src/main/resources/configuration/
>>>>>>> standalone/subsystems-cloud.xml#L19
>>>>>>>
>>>>>>> The default configuration XML doesn't contain the `node-identifier`
>>>>>>> attribute. I can add it manually in the cloud.xml but I believe the
>>>>>>> right
>>>>>>> approach is to modify the transaction subsystem.
>>>>>>>
>>>>>>>
>>>>>>> > Rado
>>>>>>> >
>>>>>>> > > Thanks,
>>>>>>> > > Seb
>>>>>>> > >
>>>>>>> > > [1] usually set to node-identifier="${jboss.node.name}"
>>>>>>> > >
>>>>>>> > >
>>>>>>>
>>>>>> > > On Mon, Apr 9, 2018 at 10:39 AM Sanne Grinovero <sanne at
>>>>>>> infinispan.org>
>>>>>>> > > wrote:
>>>>>>> > >>
>>>>>>> > >> On 9 April 2018 at 09:26, Sebastian Laskawiec <slaskawi at
>>>>>>> redhat.com>
>>>>>>
>>>>>>
>>>>>>> > wrote:
>>>>>>> > >> > Thanks for looking into it Sanne. Of course, we should add it
>>>>>>> (it can
>>>>>>> > be
>>>>>>> > >> > set
>>>>>>> > >> > to the same name as hostname since those are unique in
>>>>>>> Kubernetes).
>>>>>>> > >> >
>>>>>>> > >> > Created https://issues.jboss.org/browse/ISPN-9051 for it.
>>>>>>> > >> >
>>>>>>> > >> > Thanks again!
>>>>>>> > >> > Seb
>>>>>>> > >>
>>>>>>> > >> Thanks Sebastian!
>>>>>>> > >>
>>>>>>> > >> >
>>>>>>>
>>>>>> > >> > On Fri, Apr 6, 2018 at 8:53 PM Sanne Grinovero <sanne at
>>>>>>> infinispan.org>
>>>>>>
>>>>>>
>>>>>>> > >> > wrote:
>>>>>>> > >> >>
>>>>>>> > >> >> Hi all,
>>>>>>> > >> >>
>>>>>>> > >> >> I've started to use the Infinispan Openshift Template and was
>>>>>>> > browsing
>>>>>>> > >> >> through the errors and warnings this produces.
>>>>>>> > >> >>
>>>>>>> > >> >> In particular I noticed "WFLYTX0013: Node identifier
>>>>>>> property is set
>>>>>>> > >> >> to the default value. Please make sure it is unique." being
>>>>>>> produced
>>>>>>> > >> >> by the transaction system.
>>>>>>> > >> >>
>>>>>>> > >> >> The node id is usually not needed for developer's
>>>>>>> convenience and
>>>>>>> > >> >> assuming there's a single node in "dev mode", yet clearly the
>>>>>>> > >> >> Infinispan template is meant to work with multiple nodes
>>>>>>> running so
>>>>>>> > >> >> this warning seems concerning.
>>>>>>> > >> >>
>>>>>>> > >> >> I'm not sure what the impact is on the transaction manager
>>>>>>> so I asked
>>>>>>> > >> >> on the Narayana forums; Tom pointed me to some thourough
>>>>>>> design
>>>>>>> > >> >> documents and also suggested the EAP image does set the node
>>>>>>> > >> >> identifier:
>>>>>>> > >> >>  - https://developer.jboss.org/message/981702#981702
>>>>>>> > >> >>
>>>>>>> > >> >> WDYT? we probably want the Infinispan template to set this
>>>>>>> as well,
>>>>>>> > or
>>>>>>> > >> >> silence the warning?
>>>>>>> > >> >>
>>>>>>> > >> >> Thanks,
>>>>>>> > >> >> Sanne
>>>>>>> > >> >> _______________________________________________
>>>>>>> > >> >> infinispan-dev mailing list
>>>>>>>
>>>>>> > >> >> infinispan-dev at lists.jboss.org
>>>>>>
>>>>>>
>>>>>>> > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> > >> >
>>>>>>> > >> >
>>>>>>> > >> > _______________________________________________
>>>>>>> > >> > infinispan-dev mailing list
>>>>>>>
>>>>>> > >> > infinispan-dev at lists.jboss.org
>>>>>>
>>>>>>
>>>>>>> > >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> > >> _______________________________________________
>>>>>>> > >> infinispan-dev mailing list
>>>>>>> > >> infinispan-dev at lists.jboss.org
>>>>>>> > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> >
>>>>>>> -------------- next part --------------
>>>>>>> An HTML attachment was scrubbed...
>>>>>>> URL: http://lists.jboss.org/pipermail/wildfly-dev/
>>>>>>> attachments/20180416/65962cf1/attachment-0001.html
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>



-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20180507/2a70ac8a/attachment-0001.html

More information about the wildfly-dev mailing list