[wildfly-dev] [infinispan-dev] WFLYTX0013 in the Infinispan Openshift Template

Sebastian Laskawiec slaskawi at redhat.com
Tue Jul 10 09:32:58 EDT 2018 

Hey Tom!

Could you please tell me what is the status of this? It seems one of the
Keycloak users got hit by this problem:
http://lists.jboss.org/pipermail/keycloak-dev/2018-July/010985.html

Thanks,
Sebastian

On Thu, May 10, 2018 at 11:10 AM Tom Jenkinson <tom.jenkinson at redhat.com>
wrote:

> Sure - thanks!
>
> On 10 May 2018 at 02:56, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>
>> I'm sorry for delay... I got sucked by the Summit prep activities.
>>
>> Yes, to all, what you said! Shall I create a JIRA for you?
>>
>>
>>
>> On Wed, May 9, 2018 at 9:39 AM Tom Jenkinson <tom.jenkinson at redhat.com>
>> wrote:
>>
>>> Thanks Brian. Does it work for you Sebastian?
>>>
>>> On 8 May 2018 at 23:05, Brian Stansberry <brian.stansberry at redhat.com>
>>> wrote:
>>>
>>>> Ah, ok. I was thinking of scripting in the broad sense the various
>>>> stuff that goes into creating images.
>>>>
>>>> In any case, I don't see any downside to having node-identifier="${
>>>> jboss.tx.node.id:1}" in the standard WF config files.
>>>>
>>>>
>>>>
>>>> On Tue, May 8, 2018 at 3:07 PM, Tom Jenkinson <tom.jenkinson at redhat.com
>>>> > wrote:
>>>>
>>>>> I think they want to avoid changing the standalone.xml file and just
>>>>> want to control it from their startup script.
>>>>>
>>>>> On 8 May 2018 at 18:45, Brian Stansberry <brian.stansberry at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> I might have missed something along the way, but if they are going to
>>>>>> do scripting wouldn't they just set the attribute to ${
>>>>>> jboss.node.name} and count on the fact that this is unique per pod?
>>>>>>
>>>>>> On Tue, May 8, 2018 at 3:28 AM, Tom Jenkinson <
>>>>>> tom.jenkinson at redhat.com> wrote:
>>>>>>
>>>>>>> Thanks for confirming Brian.
>>>>>>>
>>>>>>> Perhaps we could set it to:
>>>>>>> node-identifier="${jboss.tx.node.id:1}"
>>>>>>> (a bit like
>>>>>>> https://github.com/jboss-developer/jboss-eap-quickstarts/tree/7.1/jts
>>>>>>> )
>>>>>>>
>>>>>>> Sebastian could set -Djboss.tx.node.id during startup in a script?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 7 May 2018 at 22:08, Brian Stansberry <
>>>>>>> brian.stansberry at redhat.com> wrote:
>>>>>>>
>>>>>>>> If it's not already set, WildFly sets the system property
>>>>>>>> jboss.node.name at the very beginning of server boot, so ${
>>>>>>>> jboss.node.name*:1*} would not resolve to 1.
>>>>>>>>
>>>>>>>> On Sun, May 6, 2018 at 6:10 PM, Sebastian Laskawiec <
>>>>>>>> slaskawi at redhat.com> wrote:
>>>>>>>>
>>>>>>>>> Ok, so how about doing the same thing you suggested, but just more
>>>>>>>>> explicitly - adding node-identifier="${jboss.node.name*:1*}".
>>>>>>>>> This way the bare metal deployment should be happy (since the default is
>>>>>>>>> still 1) and we wouldn't need to override it in Infinispan.
>>>>>>>>>
>>>>>>>>> On Tue, May 1, 2018 at 10:09 AM Tom Jenkinson <
>>>>>>>>> tom.jenkinson at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> I am not sure - the default should be "1" for the bare metal case
>>>>>>>>>> so the warning is reliably triggered but the default can be the pod name
>>>>>>>>>> for OpenShift templates that only allow a single instance of the
>>>>>>>>>> application server - does that help?
>>>>>>>>>>
>>>>>>>>>> The file you looked to want to edit is shared by bare metal and
>>>>>>>>>> other deployment environments so it would be confusing to set the default
>>>>>>>>>> to jboss.node.name there IMO.
>>>>>>>>>>
>>>>>>>>>> On 1 May 2018 at 03:39, Sebastian Laskawiec <slaskawi at redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Fair enough Tom. Thanks for explanation.
>>>>>>>>>>>
>>>>>>>>>>> One more request - would you guys be OK with me adding
>>>>>>>>>>> a node-identifier="${jboss.node.name}" to the transaction
>>>>>>>>>>> subsystem template [1]? This way we wouldn't need to copy it into
>>>>>>>>>>> Infinispan (since we need to set it).
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> https://github.com/wildfly/wildfly/blob/master/transactions/src/main/resources/subsystem-templates/transactions.xml#L6
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 18, 2018 at 3:37 PM Tom Jenkinson <
>>>>>>>>>>> tom.jenkinson at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On 18 April 2018 at 14:07, Sebastian Laskawiec <
>>>>>>>>>>>> slaskawi at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Tom,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Comments inlined.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Apr 17, 2018 at 4:37 PM Tom Jenkinson <
>>>>>>>>>>>>> tom.jenkinson at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 16 April 2018 at 09:31, <> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Adding +WildFly Dev <wildfly-dev at lists.jboss.org> to the
>>>>>>>>>>>>>>> loop
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the explanation Rado.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> TL;DR: A while ago Sanne pointed out that we do not set
>>>>>>>>>>>>>>> `node-identifier`
>>>>>>>>>>>>>>> in transaction subsystem by default. The default value for
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> `node-identifier` attribute it `1`. Not setting this
>>>>>>>>>>>>>>> attribute might cause
>>>>>>>>>>>>>>> problems in transaction recovery. Perhaps we could follow
>>>>>>>>>>>>>>> Rado's idea and
>>>>>>>>>>>>>>> set it to node name by default?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Indeed - it would cause serious data integrity problems if a
>>>>>>>>>>>>>> non-unique node-identifier is used.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Some more comments inlined.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Apr 13, 2018 at 7:07 PM Radoslav Husar <rhusar at
>>>>>>>>>>>>>>> redhat.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > Hi Sebastian,
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > On Wed, Apr 11, 2018 at 2:31 PM, Sebastian Laskawiec
>>>>>>>>>>>>>>> > <slaskawi at redhat.com> wrote:
>>>>>>>>>>>>>>> > > Hey Rado, Paul,
>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>> > > I started looking into this issue and it turned out that
>>>>>>>>>>>>>>> WF subsystem
>>>>>>>>>>>>>>> > > template doesn't provide `node-identifier` attribute [1].
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > I assume you mean that the default WildFly server profiles
>>>>>>>>>>>>>>> do not
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > explicitly define the attribute. Right ? thus the value
>>>>>>>>>>>>>>> defaults in
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > the model to "1"
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> https://github.com/wildfly/wildfly/blob/master/transactions/src/main/java/org/jboss/as/txn/subsystem/TransactionSubsystemRootResourceDefinition.java#L97
>>>>>>>>>>>>>>> > which sole intention seems to be to log a warning on boot
>>>>>>>>>>>>>>> if the value
>>>>>>>>>>>>>>> > is unchanged.
>>>>>>>>>>>>>>> > Why they decided on a constant that will be inherently not
>>>>>>>>>>>>>>> unique as
>>>>>>>>>>>>>>> > opposed to defaulting to the node name (which we already
>>>>>>>>>>>>>>> require to be
>>>>>>>>>>>>>>> > unique) as clustering node name or undertow instance-id
>>>>>>>>>>>>>>> does, is
>>>>>>>>>>>>>>> > unclear to me.
>>>>>>>>>>>>>>> > Some context is on
>>>>>>>>>>>>>>> https://issues.jboss.org/browse/WFLY-1119.
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In OpenShift environment we could set it to `hostname`. This
>>>>>>>>>>>>>>> is guaranteed
>>>>>>>>>>>>>>> to be unique in whole OpenShift cluster.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We do this too in EAP images.
>>>>>>>>>>>>>> To Rado's point, the default is "1" so we can print the
>>>>>>>>>>>>>> warning to alert people they are misconfigured - it seems to be working :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is the point. From my understanding, if we set it to node
>>>>>>>>>>>>> name (instead of "1"), we could make it always work correctly. We could
>>>>>>>>>>>>> even remove the code that emits the warning (since the node name needs to
>>>>>>>>>>>>> be unique).
>>>>>>>>>>>>>
>>>>>>>>>>>>> To sum it up - if we decided to proceed this way, there would
>>>>>>>>>>>>> be no requirement of setting the node-identifier at all.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> For OpenShift you are right there is no requirement for someone
>>>>>>>>>>>> to change the node-identifier from the podname and so that is why EAP
>>>>>>>>>>>> images do that.
>>>>>>>>>>>>
>>>>>>>>>>>> For bare-metal it is different as there can be two servers on
>>>>>>>>>>>> the same machine so they were configured to use the hostname as they
>>>>>>>>>>>> node-identifier then if they were also connected to the same resource
>>>>>>>>>>>> managers or the same object store they would interfere with each other.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > > I'm not sure if you guys are the right people to ask,
>>>>>>>>>>>>>>> but is it safe to
>>>>>>>>>>>>>>> > > leave it set to default? Or shall I override our
>>>>>>>>>>>>>>> Infinispan templates and
>>>>>>>>>>>>>>> > > add this parameter (as I mentioned before, in OpenShift
>>>>>>>>>>>>>>> this I wanted to
>>>>>>>>>>>>>>> > set
>>>>>>>>>>>>>>> > > it as Pod name trimmed to the last 23 chars since this
>>>>>>>>>>>>>>> is the limit).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Putting a response to this in line - I am not certain who
>>>>>>>>>>>>>> originally proposed this.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You must use a globally unique node-identifier. If you are
>>>>>>>>>>>>>> certain the last 23 characters guarantee that it would be valid - if there
>>>>>>>>>>>>>> is a chance they are not unique it is not valid to trim.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> If that's not an issue, again, we could use the same limit as
>>>>>>>>>>>>> we have for node name.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > It is not safe to leave it set to "1" as that results in
>>>>>>>>>>>>>>> inconsistent
>>>>>>>>>>>>>>> > processing of transaction recovery.
>>>>>>>>>>>>>>> > IIUC we already set it to the node name for both EAP and
>>>>>>>>>>>>>>> JDG
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> https://github.com/jboss-openshift/cct_module/blob/master/os-eap70-openshift/added/standalone-openshift.xml#L411
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> https://github.com/jboss-openshift/cct_module/blob/master/os-jdg7-conffiles/added/clustered-openshift.xml#L282
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > which in turn defaults to the pod name ? so which profiles
>>>>>>>>>>>>>>> are we
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > talking about here?
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Granted, we set it by default in CCT Modules. However in
>>>>>>>>>>>>>>> Infinispan we just
>>>>>>>>>>>>>>> grab provided transaction subsystem when rendering full
>>>>>>>>>>>>>>> configuration from
>>>>>>>>>>>>>>> featurepacks:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://github.com/infinispan/infinispan/blob/master/server/integration/feature-pack/src/main/resources/configuration/standalone/subsystems-cloud.xml#L19
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The default configuration XML doesn't contain the
>>>>>>>>>>>>>>> `node-identifier`
>>>>>>>>>>>>>>> attribute. I can add it manually in the cloud.xml but I
>>>>>>>>>>>>>>> believe the right
>>>>>>>>>>>>>>> approach is to modify the transaction subsystem.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > Rado
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > > Thanks,
>>>>>>>>>>>>>>> > > Seb
>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>> > > [1] usually set to node-identifier="${jboss.node.name}"
>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > > On Mon, Apr 9, 2018 at 10:39 AM Sanne Grinovero <sanne at
>>>>>>>>>>>>>>> infinispan.org>
>>>>>>>>>>>>>>> > > wrote:
>>>>>>>>>>>>>>> > >>
>>>>>>>>>>>>>>> > >> On 9 April 2018 at 09:26, Sebastian Laskawiec <slaskawi
>>>>>>>>>>>>>>> at redhat.com>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>> > >> > Thanks for looking into it Sanne. Of course, we
>>>>>>>>>>>>>>> should add it (it can
>>>>>>>>>>>>>>> > be
>>>>>>>>>>>>>>> > >> > set
>>>>>>>>>>>>>>> > >> > to the same name as hostname since those are unique
>>>>>>>>>>>>>>> in Kubernetes).
>>>>>>>>>>>>>>> > >> >
>>>>>>>>>>>>>>> > >> > Created https://issues.jboss.org/browse/ISPN-9051
>>>>>>>>>>>>>>> for it.
>>>>>>>>>>>>>>> > >> >
>>>>>>>>>>>>>>> > >> > Thanks again!
>>>>>>>>>>>>>>> > >> > Seb
>>>>>>>>>>>>>>> > >>
>>>>>>>>>>>>>>> > >> Thanks Sebastian!
>>>>>>>>>>>>>>> > >>
>>>>>>>>>>>>>>> > >> >
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > >> > On Fri, Apr 6, 2018 at 8:53 PM Sanne Grinovero <sanne
>>>>>>>>>>>>>>> at infinispan.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > >> > wrote:
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> > >> >> Hi all,
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> > >> >> I've started to use the Infinispan Openshift
>>>>>>>>>>>>>>> Template and was
>>>>>>>>>>>>>>> > browsing
>>>>>>>>>>>>>>> > >> >> through the errors and warnings this produces.
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> > >> >> In particular I noticed "WFLYTX0013: Node identifier
>>>>>>>>>>>>>>> property is set
>>>>>>>>>>>>>>> > >> >> to the default value. Please make sure it is
>>>>>>>>>>>>>>> unique." being produced
>>>>>>>>>>>>>>> > >> >> by the transaction system.
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> > >> >> The node id is usually not needed for developer's
>>>>>>>>>>>>>>> convenience and
>>>>>>>>>>>>>>> > >> >> assuming there's a single node in "dev mode", yet
>>>>>>>>>>>>>>> clearly the
>>>>>>>>>>>>>>> > >> >> Infinispan template is meant to work with multiple
>>>>>>>>>>>>>>> nodes running so
>>>>>>>>>>>>>>> > >> >> this warning seems concerning.
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> > >> >> I'm not sure what the impact is on the transaction
>>>>>>>>>>>>>>> manager so I asked
>>>>>>>>>>>>>>> > >> >> on the Narayana forums; Tom pointed me to some
>>>>>>>>>>>>>>> thourough design
>>>>>>>>>>>>>>> > >> >> documents and also suggested the EAP image does set
>>>>>>>>>>>>>>> the node
>>>>>>>>>>>>>>> > >> >> identifier:
>>>>>>>>>>>>>>> > >> >>  - https://developer.jboss.org/message/981702#981702
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> > >> >> WDYT? we probably want the Infinispan template to
>>>>>>>>>>>>>>> set this as well,
>>>>>>>>>>>>>>> > or
>>>>>>>>>>>>>>> > >> >> silence the warning?
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> > >> >> Thanks,
>>>>>>>>>>>>>>> > >> >> Sanne
>>>>>>>>>>>>>>> > >> >> _______________________________________________
>>>>>>>>>>>>>>> > >> >> infinispan-dev mailing list
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > >> >> infinispan-dev at lists.jboss.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > >> >>
>>>>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>>>>>>>> > >> >
>>>>>>>>>>>>>>> > >> >
>>>>>>>>>>>>>>> > >> > _______________________________________________
>>>>>>>>>>>>>>> > >> > infinispan-dev mailing list
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > >> > infinispan-dev at lists.jboss.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > >> >
>>>>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>>>>>>>> > >> _______________________________________________
>>>>>>>>>>>>>>> > >> infinispan-dev mailing list
>>>>>>>>>>>>>>> > >> infinispan-dev at lists.jboss.org
>>>>>>>>>>>>>>> > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> -------------- next part --------------
>>>>>>>>>>>>>>> An HTML attachment was scrubbed...
>>>>>>>>>>>>>>> URL:
>>>>>>>>>>>>>>> http://lists.jboss.org/pipermail/wildfly-dev/attachments/20180416/65962cf1/attachment-0001.html
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> wildfly-dev mailing list
>>>>>>>>> wildfly-dev at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Brian Stansberry
>>>>>>>> Manager, Senior Principal Software Engineer
>>>>>>>> Red Hat
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Brian Stansberry
>>>>>> Manager, Senior Principal Software Engineer
>>>>>> Red Hat
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Brian Stansberry
>>>> Manager, Senior Principal Software Engineer
>>>> Red Hat
>>>>
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20180710/d622fe38/attachment-0001.html

More information about the wildfly-dev mailing list