Supporting OpenTelemetry in Reactive Messaging connectors
by Kabir Khan
Hi,
This is about the following RFEs:
https://issues.redhat.com/browse/WFLY-19835
https://issues.redhat.com/browse/WFLY-19836
They are about integrating the AMQP and Kafka connectors used by Reactive
Messaging with OpenTelemetry tracing. I'd just like to brainstorm a bit
before jumping in.
The code is already there in SmallRye but is totally disabled in WildFly
via this ConfigSource setting the
mp.messaging.connector.smallrye-amqp.tracing-enabled and
mp.messaging.connector.smallrye-kafka.tracing-enabled MP Config properties
to false.
https://github.com/wildfly/wildfly/blob/main/microprofile/reactive-messag...
This config source has the max ordinal
<https://github.com/wildfly/wildfly/blob/main/microprofile/reactive-messag...>,
so it cannot be overridden at the moment.
I think to enable this I need to:
1) Add optional module dependencies from the connector modules to the Otel
module providing Tracer
2) Adjust the properties to allow for tracing to happen
I think it makes sense to keep the default behaviour as not enabling
tracing for backwards compatibility?
To be able to enable tracing via MP Config, we would need to lower the
ordinal of the config source providing these properties. This way
deployments will be able to opt in to using tracing.
However, if tracing is enabled and the OTel modules are not present, the
SmallRye connectors will fail.
So it seems I need to add an attribute to the currently empty subsystem to
allow for tracing, and then do some capability/Galleon wiring to provision
OTel if it indicates that we allow tracing. The value of the attribute
would then influence the value of the ConfigSource properties behind the
scenes.
Since OTel being present is a yes/no situation, we could get away with one
attribute - but perhaps it makes sense to allow configuration for this for
each connector separately. Not sure how that works with the provisioning,
but I guess if both of these depend on OTel, and one of them is set to
indicate Otel is needed, we're good.
As for the values of the attributes, I am thinking:
* never - this sets the properties in the config source as it does at
present, where it is set to false with a max ordinal so deployments can not
override it
* off - this would set the config source properties to false again, but
with an ordinal value that can be overridden in the deployment (and system
properties, env vars etc). So it is off by default but can be turned on
* on - this would set the config source properties to true, but with an
ordinal value that can be overridden in the deployment (and system
properties, env vars etc). So it is on by default but can be turned off
* always - this is like 'never' so the config source ordinal is the
maximum, and the properties are set to true. Thus deployments can not opt
out.
Does the capability stuff work so that if the value of the subsystem
attributes are *off*, *on* or *always* I add the requirement? And I don't
if it is *never*?
Any other thoughts/suggestions/objections?
Thanks,
Kabir
2 days, 8 hours
WildFly Feature Process Update
by Brian Stansberry
Hi,
Over the summer a number of us (primarily James Perkins, Jeff Mesnil and
Darran Lofthouse) have been working on various types of tooling related to
managing the WildFly Feature Development Process[1]. Now that feature dev
work for WF 34 is complete we are rolling out some of what we’ve been doing
so we can benefit from some automation during work on 35.
1) Updates to the overall Feature Process document
We have significantly beefed up the WildFly Community Feature Process
document
<https://github.com/wildfly/wildfly-proposals/blob/main/FEATURE_PROCESS.adoc>
to much more completely describe the process the developers should follow
when adding new features to WildFly or promoting existing features to
higher stability levels.
The rest of this message touches on things more fully described in that
document.
2) New feature planning board
We’ve added a GitHub project to the wildfly GitHub org that we will use to
track feature development work:
https://github.com/orgs/wildfly/projects/7/
Each feature undergoing development will be tracked by a GitHub issue tied
to this project (more on those issues in the next section).
In the past the set of issues that were in-flight for a WildFly release
have either been tracked on the back of a napkin (by me) or lately in a
spreadsheet (by the more organized Darran). Both approaches are inadequate
and are hidden from the community. A GitHub project seems to meet our needs
nicely.[2]
Feature teams should use the project to update the overall status of their
issues:
https://github.com/orgs/wildfly/projects/7/views/2
Feature teams or the coordinator for a release can ‘schedule’ issues by
assigning an “Development Window”. We’ll create ‘windows’ for each WF
feature release, starting with 35 and 36. A window for release N will run
from the date after the Beta release of N -1 through the intended Beta
release date for N.
https://github.com/orgs/wildfly/projects/7/views/3
Scheduling is not a commitment; it’s just a way of keeping track of which
issues are being worked and may land in a given release.
3) New issue tracker
We’re using GitHub issues in the wildfly-proposals repo as the source of
issues for the WildFly Feature Planning project.
Yes, this means feature teams will need to file another issue and associate
it with development of the feature. But we want this to be really
lightweight for feature teams. Just go to
https://github.com/wildfly/wildfly-proposals/issues/new/choose and select
“Feature Development”. You get a very simple form:
https://github.com/wildfly/wildfly-proposals/issues/new?assignees=&labels...
Note that you can also navigate to this form the ‘Add item’ link in
https://github.com/orgs/wildfly/projects/7/views/1.
4) Using formal metadata fields (asciidoc attributes) in feature analyses
A key goal of our tooling is to avoid wasting contributors’ time doing
redundant entry of information related to a feature. It seems like the
primary source of data for tracking development of a feature has been the
feature analysis document itself. So we want to try and capture as much
information as possible in the analyses and then have other tools extract
and use that data.
The data we need is mostly already required in the analyses, but up to now
we provide it via raw asciidoc text. What’s new is instead for data that we
want our tooling to process we are going to require that the information
instead be provided using asciidoc attributes at the top of the document.
We’ve just merged updates to wildfly-proposals such that a GitHub action
will check for the presence of required attributes and label the PR if any
are missing. We’ve been doing this for a while checking for the ‘category’
attribute. We’re now checking for two additional attributes:
* stability-level -- the feature stability level
* issue -- a link to the GitHub issue discussed above.
If you send up a new analysis PR or update an existing one you are likely
going to have your PR flagged up by this automation as missing required
attributes. Please update the PR to conform to the new way of presenting
data.
It is not necessary to include information present in these attributes in
the document text. We’ve updated the feature analysis template[3] to remove
the suggestion to include this data in the text and instead explain the
need for the attributes.
Note that even though this information only appears in the analysis file in
asciidoc attributes, that doesn’t mean it will be hidden from users once
the analysis is merged. We’ve added logic to the Jeykll tooling that
renders the analysis asciidoc files such that it adds the relevant data to
the rendered documents we provide to users.[4] We also add labels to the
analysis PRs to show their stability level.
We’ve also added three other attributes, but haven’t added tooling yet
related to them. So we’re not verifying their use using any automation. But
the feature process itself does require use of the ‘feature-team’ attribute.
These three attributes are:
* feature-team -- this is a complex yaml structure where you provide the
github ids of the members of the feature team.
* promotes -- if the analysis is for the promotion of an existing feature
to a higher stability level, provide the issue link for the existing
feature.
* promoted-by -- We’d use this to update an already merged analysis if some
subsequent issue promotes it to a higher stability.
We’ll add tooling that uses ‘promotes’ and ‘promoted-by’ to help us track
issue promotion.
Your comments are most welcome, either here or in zulip. For some
background there’s an existing thread where we’ve been discussing this:
https://wildfly.zulipchat.com/#narrow/stream/174184-wildfly-developers/to...
Best regards,
Brian
[1] http://docs.wildfly.org/wildfly-proposals/FEATURE_PROCESS.html
[2] Why not use JIRA?
* The primary project for where a feature gets integrated varies; sometimes
WFLY, sometimes WFCORE, sometimes HAL, sometimes other.
* Often there really isn’t a primary project.
* Even if there was always one ‘right’ JIRA project it’s not clear that we
could customize it the way we want.
[3]
https://github.com/wildfly/wildfly-proposals/blob/main/design-doc-templat...
[4] https://docs.wildfly.org/wildfly-proposals/
2 days, 9 hours
WildFly 34 Schedule
by Brian Stansberry
We're aiming to have WildFly 34.0.0.Final available on wildfly.org on
Thursday, October 10. The beta should be available on September 26.
Following are the key dates:
Wed Sep 18 -- Feature Freeze
Mon Sep 23 -- WF Core Beta release
Wed Sep 25 -- Tag/deploy 34 Beta
Thu Sep 26 -- Announce release
Wed Oct 2 -- Final PRs due
Mon Oct 7 -- WF Core Final release
Wed Oct 9 -- Tag/deploy 34 Final
Thu Oct 10 -- Release available on wildfly.org
Fri Oct 11 -- Post release deliverables
Best regards,
--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
WildFly Project Lead
He/Him/His
2 days, 12 hours
WF 34 as the final release with SE 11 support?
by Brian Stansberry
WDYT?
We've been considering doing this for a long time, and I think it's only a
matter of time before important libraries needed for standard WF are only
compiled to SE 17.
I think WF 35 would be the absolute end of the line for SE 11 support in
std WF; after that Jakarta EE and probably our MP impls from smallrye will
force a move.
But I suspect something could easily happen during the WF 35 dev cycle that
forces a move, and I think we'd be better off announcing a plan to move on
around the team of WF 34 Beta, and then announcing WF 34 is the end for SE
11 when we release the Final.
Best regards,
--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
WildFly Project Lead
He/Him/His
4 days, 8 hours
