[wildfly-dev] [infinispan-dev] WFLYTX0013 in the Infinispan Openshift Template

Tue May 8 13:45:37 EDT 2018

I might have missed something along the way, but if they are going to do
scripting wouldn't they just set the attribute to ${jboss.node.name} and
count on the fact that this is unique per pod?

On Tue, May 8, 2018 at 3:28 AM, Tom Jenkinson <tom.jenkinson at redhat.com>
wrote:

> Thanks for confirming Brian.
>
> Perhaps we could set it to:
> node-identifier="${jboss.tx.node.id:1}"
> (a bit like https://github.com/jboss-developer/jboss-eap-
> quickstarts/tree/7.1/jts)
>
> Sebastian could set -Djboss.tx.node.id during startup in a script?
>
>
>
> On 7 May 2018 at 22:08, Brian Stansberry <brian.stansberry at redhat.com>
> wrote:
>
>> If it's not already set, WildFly sets the system property jboss.node.name
>> at the very beginning of server boot, so ${jboss.node.name*:1*} would
>> not resolve to 1.
>>
>> On Sun, May 6, 2018 at 6:10 PM, Sebastian Laskawiec <slaskawi at redhat.com>
>> wrote:
>>
>>> Ok, so how about doing the same thing you suggested, but just more
>>> explicitly - adding node-identifier="${jboss.node.name*:1*}". This way
>>> the bare metal deployment should be happy (since the default is still 1)
>>> and we wouldn't need to override it in Infinispan.
>>>
>>> On Tue, May 1, 2018 at 10:09 AM Tom Jenkinson <tom.jenkinson at redhat.com>
>>> wrote:
>>>
>>>> I am not sure - the default should be "1" for the bare metal case so
>>>> the warning is reliably triggered but the default can be the pod name for
>>>> OpenShift templates that only allow a single instance of the application
>>>> server - does that help?
>>>>
>>>> The file you looked to want to edit is shared by bare metal and other
>>>> deployment environments so it would be confusing to set the default to
>>>> jboss.node.name there IMO.
>>>>
>>>> On 1 May 2018 at 03:39, Sebastian Laskawiec <slaskawi at redhat.com>
>>>> wrote:
>>>>
>>>>> Fair enough Tom. Thanks for explanation.
>>>>>
>>>>> One more request - would you guys be OK with me adding
>>>>> a node-identifier="${jboss.node.name}" to the transaction subsystem
>>>>> template [1]? This way we wouldn't need to copy it into Infinispan (since
>>>>> we need to set it).
>>>>>
>>>>> [1] https://github.com/wildfly/wildfly/blob/master/transacti
>>>>> ons/src/main/resources/subsystem-templates/transactions.xml#L6
>>>>>
>>>>> On Wed, Apr 18, 2018 at 3:37 PM Tom Jenkinson <
>>>>> tom.jenkinson at redhat.com> wrote:
>>>>>
>>>>>> On 18 April 2018 at 14:07, Sebastian Laskawiec <slaskawi at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey Tom,
>>>>>>>
>>>>>>> Comments inlined.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Sebastian
>>>>>>>
>>>>>>> On Tue, Apr 17, 2018 at 4:37 PM Tom Jenkinson <
>>>>>>> tom.jenkinson at redhat.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 16 April 2018 at 09:31, <> wrote:
>>>>>>>>
>>>>>>>>> Adding +WildFly Dev <wildfly-dev at lists.jboss.org> to the loop
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for the explanation Rado.
>>>>>>>>>
>>>>>>>>> TL;DR: A while ago Sanne pointed out that we do not set
>>>>>>>>> `node-identifier`
>>>>>>>>> in transaction subsystem by default. The default value for the
>>>>>>>>> `node-identifier` attribute it `1`. Not setting this attribute
>>>>>>>>> might cause
>>>>>>>>> problems in transaction recovery. Perhaps we could follow Rado's
>>>>>>>>> idea and
>>>>>>>>> set it to node name by default?
>>>>>>>>>
>>>>>>>> Indeed - it would cause serious data integrity problems if a
>>>>>>>> non-unique node-identifier is used.
>>>>>>>>
>>>>>>>>> Some more comments inlined.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Sebastian
>>>>>>>>>
>>>>>>>>> On Fri, Apr 13, 2018 at 7:07 PM Radoslav Husar <rhusar at
>>>>>>>>> redhat.com> wrote:
>>>>>>>>>
>>>>>>>>> > Hi Sebastian,
>>>>>>>>> >
>>>>>>>>> > On Wed, Apr 11, 2018 at 2:31 PM, Sebastian Laskawiec
>>>>>>>>> > <slaskawi at redhat.com> wrote:
>>>>>>>>> > > Hey Rado, Paul,
>>>>>>>>> > >
>>>>>>>>> > > I started looking into this issue and it turned out that WF
>>>>>>>>> subsystem
>>>>>>>>> > > template doesn't provide `node-identifier` attribute [1].
>>>>>>>>> >
>>>>>>>>> > I assume you mean that the default WildFly server profiles do not
>>>>>>>>>
>>>>>>>> > explicitly define the attribute. Right ? thus the value defaults
>>>>>>>>> in
>>>>>>>>
>>>>>>>>
>>>>>>>>> > the model to "1"
>>>>>>>>> >
>>>>>>>>> > https://github.com/wildfly/wildfly/blob/master/transactions/
>>>>>>>>> src/main/java/org/jboss/as/txn/subsystem/TransactionSubsyste
>>>>>>>>> mRootResourceDefinition.java#L97
>>>>>>>>> > which sole intention seems to be to log a warning on boot if the
>>>>>>>>> value
>>>>>>>>> > is unchanged.
>>>>>>>>> > Why they decided on a constant that will be inherently not
>>>>>>>>> unique as
>>>>>>>>> > opposed to defaulting to the node name (which we already require
>>>>>>>>> to be
>>>>>>>>> > unique) as clustering node name or undertow instance-id does, is
>>>>>>>>> > unclear to me.
>>>>>>>>> > Some context is on https://issues.jboss.org/browse/WFLY-1119.
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>> In OpenShift environment we could set it to `hostname`. This is
>>>>>>>>> guaranteed
>>>>>>>>> to be unique in whole OpenShift cluster.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> We do this too in EAP images.
>>>>>>>> To Rado's point, the default is "1" so we can print the warning to
>>>>>>>> alert people they are misconfigured - it seems to be working :)
>>>>>>>>
>>>>>>>
>>>>>>> This is the point. From my understanding, if we set it to node name
>>>>>>> (instead of "1"), we could make it always work correctly. We could even
>>>>>>> remove the code that emits the warning (since the node name needs to be
>>>>>>> unique).
>>>>>>>
>>>>>>> To sum it up - if we decided to proceed this way, there would be no
>>>>>>> requirement of setting the node-identifier at all.
>>>>>>>
>>>>>>
>>>>>> For OpenShift you are right there is no requirement for someone to
>>>>>> change the node-identifier from the podname and so that is why EAP images
>>>>>> do that.
>>>>>>
>>>>>> For bare-metal it is different as there can be two servers on the
>>>>>> same machine so they were configured to use the hostname as they
>>>>>> node-identifier then if they were also connected to the same resource
>>>>>> managers or the same object store they would interfere with each other.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>> > > I'm not sure if you guys are the right people to ask, but is
>>>>>>>>> it safe to
>>>>>>>>> > > leave it set to default? Or shall I override our Infinispan
>>>>>>>>> templates and
>>>>>>>>> > > add this parameter (as I mentioned before, in OpenShift this I
>>>>>>>>> wanted to
>>>>>>>>> > set
>>>>>>>>> > > it as Pod name trimmed to the last 23 chars since this is the
>>>>>>>>> limit).
>>>>>>>>>
>>>>>>>> Putting a response to this in line - I am not certain who
>>>>>>>> originally proposed this.
>>>>>>>>
>>>>>>>> You must use a globally unique node-identifier. If you are certain
>>>>>>>> the last 23 characters guarantee that it would be valid - if there is a
>>>>>>>> chance they are not unique it is not valid to trim.
>>>>>>>>
>>>>>>>
>>>>>>> If that's not an issue, again, we could use the same limit as we
>>>>>>> have for node name.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>> > It is not safe to leave it set to "1" as that results in
>>>>>>>>> inconsistent
>>>>>>>>> > processing of transaction recovery.
>>>>>>>>> > IIUC we already set it to the node name for both EAP and JDG
>>>>>>>>> >
>>>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/master/os
>>>>>>>>> -eap70-openshift/added/standalone-openshift.xml#L411
>>>>>>>>> >
>>>>>>>>> > https://github.com/jboss-openshift/cct_module/blob/master/os
>>>>>>>>> -jdg7-conffiles/added/clustered-openshift.xml#L282
>>>>>>>>>
>>>>>>>> > which in turn defaults to the pod name ? so which profiles are we
>>>>>>>>
>>>>>>>>
>>>>>>>>> > talking about here?
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>> Granted, we set it by default in CCT Modules. However in
>>>>>>>>> Infinispan we just
>>>>>>>>> grab provided transaction subsystem when rendering full
>>>>>>>>> configuration from
>>>>>>>>> featurepacks:
>>>>>>>>> https://github.com/infinispan/infinispan/blob/master/server/
>>>>>>>>> integration/feature-pack/src/main/resources/configuration/st
>>>>>>>>> andalone/subsystems-cloud.xml#L19
>>>>>>>>>
>>>>>>>>> The default configuration XML doesn't contain the `node-identifier`
>>>>>>>>> attribute. I can add it manually in the cloud.xml but I believe
>>>>>>>>> the right
>>>>>>>>> approach is to modify the transaction subsystem.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> > Rado
>>>>>>>>> >
>>>>>>>>> > > Thanks,
>>>>>>>>> > > Seb
>>>>>>>>> > >
>>>>>>>>> > > [1] usually set to node-identifier="${jboss.node.name}"
>>>>>>>>> > >
>>>>>>>>> > >
>>>>>>>>>
>>>>>>>> > > On Mon, Apr 9, 2018 at 10:39 AM Sanne Grinovero <sanne at
>>>>>>>>> infinispan.org>
>>>>>>>>> > > wrote:
>>>>>>>>> > >>
>>>>>>>>> > >> On 9 April 2018 at 09:26, Sebastian Laskawiec <slaskawi at
>>>>>>>>> redhat.com>
>>>>>>>>
>>>>>>>>
>>>>>>>>> > wrote:
>>>>>>>>> > >> > Thanks for looking into it Sanne. Of course, we should add
>>>>>>>>> it (it can
>>>>>>>>> > be
>>>>>>>>> > >> > set
>>>>>>>>> > >> > to the same name as hostname since those are unique in
>>>>>>>>> Kubernetes).
>>>>>>>>> > >> >
>>>>>>>>> > >> > Created https://issues.jboss.org/browse/ISPN-9051 for it.
>>>>>>>>> > >> >
>>>>>>>>> > >> > Thanks again!
>>>>>>>>> > >> > Seb
>>>>>>>>> > >>
>>>>>>>>> > >> Thanks Sebastian!
>>>>>>>>> > >>
>>>>>>>>> > >> >
>>>>>>>>>
>>>>>>>> > >> > On Fri, Apr 6, 2018 at 8:53 PM Sanne Grinovero <sanne at
>>>>>>>>> infinispan.org>
>>>>>>>>
>>>>>>>>
>>>>>>>>> > >> > wrote:
>>>>>>>>> > >> >>
>>>>>>>>> > >> >> Hi all,
>>>>>>>>> > >> >>
>>>>>>>>> > >> >> I've started to use the Infinispan Openshift Template and
>>>>>>>>> was
>>>>>>>>> > browsing
>>>>>>>>> > >> >> through the errors and warnings this produces.
>>>>>>>>> > >> >>
>>>>>>>>> > >> >> In particular I noticed "WFLYTX0013: Node identifier
>>>>>>>>> property is set
>>>>>>>>> > >> >> to the default value. Please make sure it is unique."
>>>>>>>>> being produced
>>>>>>>>> > >> >> by the transaction system.
>>>>>>>>> > >> >>
>>>>>>>>> > >> >> The node id is usually not needed for developer's
>>>>>>>>> convenience and
>>>>>>>>> > >> >> assuming there's a single node in "dev mode", yet clearly
>>>>>>>>> the
>>>>>>>>> > >> >> Infinispan template is meant to work with multiple nodes
>>>>>>>>> running so
>>>>>>>>> > >> >> this warning seems concerning.
>>>>>>>>> > >> >>
>>>>>>>>> > >> >> I'm not sure what the impact is on the transaction manager
>>>>>>>>> so I asked
>>>>>>>>> > >> >> on the Narayana forums; Tom pointed me to some thourough
>>>>>>>>> design
>>>>>>>>> > >> >> documents and also suggested the EAP image does set the
>>>>>>>>> node
>>>>>>>>> > >> >> identifier:
>>>>>>>>> > >> >>  - https://developer.jboss.org/message/981702#981702
>>>>>>>>> > >> >>
>>>>>>>>> > >> >> WDYT? we probably want the Infinispan template to set this
>>>>>>>>> as well,
>>>>>>>>> > or
>>>>>>>>> > >> >> silence the warning?
>>>>>>>>> > >> >>
>>>>>>>>> > >> >> Thanks,
>>>>>>>>> > >> >> Sanne
>>>>>>>>> > >> >> _______________________________________________
>>>>>>>>> > >> >> infinispan-dev mailing list
>>>>>>>>>
>>>>>>>> > >> >> infinispan-dev at lists.jboss.org
>>>>>>>>
>>>>>>>>
>>>>>>>>> > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>> > >> >
>>>>>>>>> > >> >
>>>>>>>>> > >> > _______________________________________________
>>>>>>>>> > >> > infinispan-dev mailing list
>>>>>>>>>
>>>>>>>> > >> > infinispan-dev at lists.jboss.org
>>>>>>>>
>>>>>>>>
>>>>>>>>> > >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>> > >> _______________________________________________
>>>>>>>>> > >> infinispan-dev mailing list
>>>>>>>>> > >> infinispan-dev at lists.jboss.org
>>>>>>>>> > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>> >
>>>>>>>>> -------------- next part --------------
>>>>>>>>> An HTML attachment was scrubbed...
>>>>>>>>> URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/201
>>>>>>>>> 80416/65962cf1/attachment-0001.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>>
>>
>> --
>> Brian Stansberry
>> Manager, Senior Principal Software Engineer
>> Red Hat
>>
>
>

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20180508/fff97469/attachment-0001.html 