Subsystem Inclusion Policy & Role of Feature Packs & Add-ons
by Jason Greene
Hello Everyone,
Recently there has been some confusion about how subsystems should be distributed, and whether or not they should be part of the WildFly repository.
There are three primary use-cases for distributing a subsystem.
#1 - Inclusion in an official WildFly distribution
#2 - A user installable "add-on distribution" which can be dropped on top of a WildFly Distribution (see [A])
#3 - A separate, independant, customized distribution, with a differing identity. Possibly build but not required as a layer (see [A])
If you are after #1, then the subsystem source code (defined as the portion of code which integrates with the server using the subsystem facilities) MUST be included in the WildFly repo. This is because subsystems heavily impact the stability of the server and our compliance with our strict management compatibility policy, and additionally it allows for us to keep all included subsystems up to date with core infrastructure changes such as capabilities and requirements, and the upcoming elytron security integration. Under this approach, a feature-pack is unlikely to be used, as it would likely just be part of the full feature-pack. It could very well be that we would introduce a different more expansive feature-pack in the future defining a larger distribution foot-print, however, there are currently no plans to do so.
If you are after #2, then you do not want a feature-pack, as feature-packs are just for building custom server distributions. If your use-case is #2 you are by definition not a custom server distribution, but rather a set of modules built the normal maven way.
If you are after #3, then you likely wish to use the feature-pack mechanism to make it easy to produce your custom distribution. This facility would allow you to keep your source repository limited to just the new subsystems you introduce, and pull the rest of the server bits via a maven dep. It is important, that you change the identity of the server (see [A]), such that patches for the official WildFly server are not accidentally installed.
Thanks!
[A] https://developer.jboss.org/wiki/LayeredDistributionsAndModulePathOrganiz...
--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat
7 years, 11 months
Fixing handling of 'required' attributes with 'alternatives'
by Brian Stansberry
tl;dr;
It’s not an uncommon thing to have a management attribute A that is required; i.e. must be set, but also has an alternative attribute B, where setting B satisfies the requirement for A, making an undefined value for A legal. We’re not handling that correctly and I want to fix it.[1] But fixing it will require some coordination with the HAL console.
Right now the way AttributeDefinition building works it’s not practical to declare that an attribute is required but only if no alternative is set. So instead attributes are declared as if they aren’t really required. This leads to less than helpful input validation, e.g. [2] and [3], since the attribute definition is imprecise.
The change I’d like to make will alter the read-resource-description output for 4 attributes, changing the value of the ‘nillable’ description from ‘false’ to ‘true’ so I want to coordinate that with the console team.
Long version
Harald and Claudio you guys are the main audience here. :)
For even longer version see description and comments on [1].
In a nutshell, if devs set the ‘allowNull’ property on an AttributeDefinition to ‘true’, the r-r-d output for the attribute has “nillable” => true. But there is no way to say “allowNull but only if an alternative is set.” So people are setting “allowNull” to true even if the attribute should be set in the absence of alternatives. And HAL has no metadata available to it to tell users they *must* set one of the alternatives. So I want to:
1) Add a setRequired(boolean) method to the AD builders where the fact that it means “must be defined if no alternative is defined” is explicitly declared
2) @Deprecate setAllowNull and point to setRequired
3) Clarify the meaning of setAllowNull(true) as being the same as setRequired(false)
4) Change the builder ‘allowNull’ constructor param name to “optional” and document its meaning as “allowing undefined values even in the absence of defined alternatives”. I could call the param ‘notRequired’ which is clearer in meaning but odd.
Then I will add a new ‘required’ metadata field to the r-r-d output and change the impl of the existing ’nillable’ metadata to a logical “!required || (alternatives != null && alternatives.length > 0)”
This will result in a change in the ‘nillable’ value for 4 attributes in the WildFly model from ‘false’ to ’true':
/subsystem=transactions PROCESS_ID_SOCKET_BINDING and PROCESS_ID_UUID
/core-service=management/security-realm=*/authenticaton=ldap USERNAME_FILTER and ADVANCED_FILTER
The latter two are not exposed in the console so the issue really is the two transaction attributes.
I haven’t checked other subsystems in things like Teiid or JDG, but if there are only 4 in all of WildFly I doubt there are many.
Also, if people start using the new behavior to correct problems like [2] and [3] people may expect the console to understand and reflect the concept of “required but only if there is no alternative’.
[1] https://issues.jboss.org/browse/WFCORE-1556
[2] https://issues.jboss.org/browse/WFLY-6608
[3] https://issues.jboss.org/browse/WFLY-6607
--
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat
8 years, 1 month
Making Jandex annotation index available to components
by Gunnar Morling
Hi,
---
TL,DR: Hibernate components would like to receive the Jandex index during
deployment. A new method IndexView#close() may help to address concerns
about long-living index view references
---
Different projects from the Hibernate umbrella (e.g. ORM, Validator and
Search) rely heavily on meta-data given via annotations. In order to speed
up annotation retrieval, we are looking into using the Jandex index created
for deployments by the container.
This would require subsystems like JipiJapa or the one for Bean Validation
to pass the IndexView for the deployment to the bootstrapped component. The
change itself should be easy, I hope. But Scott expressed concerns about
components keeping references to the index after the deployment phase has
succeeded. As indexes can be large, this may potentially block a big chunk
of memory for as long as the deployment is running.
Of course we can promise that Hibernate components wouldn't keep such
references ;) But if that's not enough, could this concern be mitigated by
adding a new method IndexView#close()?
This method would free all resources of the index and would be called by
the container after the deployment has finished. Any operation on a closed
index view would throw an exception indicating that the view has been
closed. Then, even if a component accidentally kept a reference to an
IndexView, it wouldn't retain much memory (of course the index view
wouldn't be of much use to the component in that state).
Would that be an acceptable way forward? If so, I could send a pull request
towards Jandex. In a next step WF would have to make use of that new
version and invoke close() on a deployment's index at a suitable time
during the deployment lifecycle and eventually pass the index to ORM et al.
In a subsequent step we may consider to address use cases such as dynamic
reconfiguration which would require to access the index *after* deployment
time. One way for addressing this may be to allow for passing in some kind
of executor which would expect e.g. a Lambda doing work on an IndexView
passed in by the executor. That way components could access the index at
runtime, while not keeping a reference to it themselves and thus allowing
the container to manage the index and e.g. close or re-open it upon
execution. But I think this could be done as a follow-up.
Any thoughts?
Thanks,
--Gunnar
PS: As a side note, Hibernate Validator currently reads annotations lazily
at runtime (when first validating a specific bean type). When using Jandex,
we'd have to change this to eagerly build up all the meta-data for the
types listed in the index passed during bootstrap. A bit of work, but as
said we hope to gain a nice performance improvement from it.
8 years, 1 month
WFLY/WFCORE PRs
by Kabir Khan
Please try to include a link to the Jira in the pull request when opening them. I want to be a good citizen and resolve them when merging, but it is a PITA to have to manually find the Jira issue in the absence of links.
8 years, 1 month
Updating components in WildFly
by Gunnar Morling
Hi,
Users of the different Hibernate projects often wish to upgrade their
WildFly installation to the latest releases of Hibernate ORM, Search or
Validator.
For that purpose we began to provide ZIP files containing the newer JARs
and module.xml descriptors, to be unzipped into the server's module
directory. We use separate slot names, so to not overwrite existing modules
but just add newer versions next to them.
But sometimes redefining the default slot actually is desirable. If an
update for instance is just a bugfix release, it's sensible to push this to
the main slot so users can benefit from it without having to explicitly
specify a slot name, suppress any implicit default module and so on.
But as I said we don't want to mess with any existing modules, so I've been
looking into alternatives. One thing I found was to provide the updated
modules in a separate layer, which then overrides the original modules from
the "base" layer, without physically altering them. The user can go back to
the original ones just by removing an entry in layers.conf.
That works nicely, but then I also learned about the patch mechanism. So I
wondered whether that should be the approach for us to follow, e.g. I also
saw that Weld is providing patches for WF. Is this then the officially
recommended way (TM) for upstream projects for shipping easy-to-use updates
of their components for WF?
One thing getting in the way of this is the lack of a Maven plug-in for the
patch-gen tool. This makes patch creation quite unwieldy in an automated
build process. Is there any chance for getting a Maven plug-in for
patch-gen? If there is interest, I'd also be volunteering to provide it, it
shouldn't take too long.
Thanks for any hints,
--Gunnar
8 years, 1 month
log4j2 Support in WildFly
by James Perkins
Hello All,
tl;dr;
Do users expect their log4j2 configurations to be picked up and used to
configure a log manager? Or do they just want to use the log4j2 API and
have the logging subsystem be the spot where the configuration is kept
Long version:
It's been requested for some time to have log4j2 support in WildFly [1].
About a month ago I was finally able to start looking into this.
In log4j2 [2] they "separated" the API from the implementation to make it
"clear for application developers which classes and methods they can use
while ensuring forward compatibility". However API has no configuration
options. It's essentially like a logging facade which requires log4j-core
[3] if you want appenders, layouts, etc.
While it was pretty easy to implement just the API [4], I'm not sure this
is what users are looking for. Implementing the API will allow you to use
log4j2 loggers which just delegate to JBoss Log Manager. However none of
the log4j2 appenders, async logging or log4j2 configurations could be used.
This would require log4j-core [3].
If users expect their log4j2 configuration files within their deployment to
configure a log4j2 LogManager we run into problems, since this would
require log4j-core to work. The simple solution seemed to be if they want
that to work they just need to include log4j-core in their deployment.
However in the awesomeness that is log4j2 (where's a SarcMark TM when you
need it?) parts of log4j-core seem to require a
org.apache.logging.log4j.core.LoggerContext instead of the LoggerContext
from the API. This means we can't simply just use our own LoggerContext
implementation and just delegate to the log4j-core embedded in the
deployment.
I guess that got into a little too much detail there :) The real question
is do we want to support user defined log4j2 configurations? If so it may
get a bit tricky getting this to work. If not we can easily support just
delegating the log4j2 loggers to JBoss Log Manager loggers.
Any feedback is appreciated. Especially if you're familiar with log4j2 and
have opinions on how it should work with WildFly.
[1]: https://issues.jboss.org/browse/WFCORE-482
[2]: http://logging.apache.org/log4j/2.x/
[3]: https://logging.apache.org/log4j/2.x/log4j-core/
[4]: https://github.com/jamezp/log4j2-jboss-logmanager
--
James R. Perkins
JBoss by Red Hat
8 years, 1 month
classloading problem prevents Wildfly from running (Wildfly 10.1, LogManager, IBM JDK8, yajsw)
by Will Milspec
Hi all,
I've run into a classloading problem that prevents wildfly 10.1 from
running with the following stack:
-Wildfly 10.1
-IBM JDK 8 (the latest, 8.0-3.12)
-YAJSW ("Yet Another Java Service Wrapper")
-Ubuntu 14.04 linux AMD 64
LogManager Classloader check fails with IBMJDK8 and YAJSW
==========================================================
This stack fails because wildfly/jboss thinks the logManager is not loaded
properly.
Wildfly concludes as such because this check returns true:
if (LogManager.getLogManager().getClass() == LogManager.class) {
To clarify: with this tech stack, the system classloader loads both
LogManager.getLogManager() and LogManager.class. Thus the two classes are
identical, i.e. "=="
I'll call this stack:
Stack1) IBMJDK8 and Yajsw
Log Manager Classloader check succeeds with other stacks
==========================================================
This 'class==" check succeeds (returns false) in all other stacks I've
tested with Wildfly 10.1
Stack2) IBMJDK8 from the command line (no yasw)
Stack3) Oracle JDK8 with yajsw
Stack4) Oracle JDK8 from the command line (no yajsw)
With stacks 2-4, the jboss-module-classloader loads
"LogManager.getLogManager()", whereas the system classloader loads
"LogManager.class"
I have no idea why the difference, i.e. why "ibmjdk8 and yajsw" uses
"system classloader" to load LogManager.getLogManager and why the
difference for the other stacks.
Note that between "Stack1" and "Stack2", I only changed "which java
command". All other configurations are identical.
Code Details: Where this check occurs
=============================
I've searched the wildfly code and found this check in two places:
jboss-modules: Main.java
wildfly-core: LoggingExtension.java
I may have missed something. (I checked the versions in the
shipped-with-wildfly-10.1 jar, cloned the various git repos and checked out
the corresponding versions for jboss-modules and wildlfy-core)
Xref, Xpost
===================
As this problem has several components, I've posted it in a few places:
yajsw forums, ibm jdk forums etc.
Yeah, I know cross-posting is bad form. Mea culpa. But as user, I did not
know (and still don't) if the problem lies in "yajsw code" or
"wildfly/jboss code" or in the IBM JDK. This is "low-level stuff"
Other posts (same information):
https://www.ibm.com/developerworks/community/forums/html/topic?id=8e9e4ae...
https://sourceforge.net/p/yajsw/discussion/810311/thread/e730451b/?limit=25
Questions
=====================
-Any workarounds? (i've tried all sorts of tweaks, all dead end)
-Any idea why the difference, i.e. why would systemclassloader load
"LogManager.getLogManager()" with "Stack1" vs "Stacks2-4"
thanks in advance!
8 years, 1 month
Early Access builds for JDK 8u122 b02 , JDK 9 & JDK 9 with Project Jigsaw b140 are available on java.net
by Rory O'Donnell
Hi Jason/Tomaz,
Early Access b02 <https://jdk8.java.net/download.html> for JDK 8u122 is
available , summary of changes are listed here.
<http://www.java.net/download/java/jdk8u122/changes/jdk8u122-b02.html>
Early Access b140 <https://jdk9.java.net/jigsaw/> (#5625) for JDK 9 with
Project Jigsaw is available on java.net, summary of changes are listed
here.
<http://www.java.net/download/java/jigsaw/archive/140/binaries/jdk-9+140.html>
Early Access b140 <https://jdk9.java.net/download/> for JDK 9 is
available on java.net, summary of changes are listed here
<http://www.java.net/download/java/jdk9/changes/jdk-9+140.html>.
A couple of items to point out with regard to b140:
1. A fix for Cyclic interface initialization causes JVM crash is
included in b140
2. We are requesting feedback on a change that went into JDK 9 b140
The java.io.FilePermission class was changed to remove pathname
canonicalization from its creation,
along with a system property to revert the behavior back to way it
worked in the previous JDK release.
We do this mainly for performance enhancement so that there is no
need to consult the file system every
time a FilePermisson is created. If you use a security manager and
file permissions then you should read
the details as described on the jdk9 mailing list [1]
Feedback is requested via core-libs-dev(a)openjdk.java.net
The new GA date of JDK 9 has been updated on the JDK 9 Project page [2],
further details in Mark Reinhold's email [3].
Rgds,Rory
[1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/005062.html
[2] http://openjdk.java.net/projects/jdk9/
[3]
http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-October/005092.html
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA, Dublin,Ireland
8 years, 2 months
selecting packages from feature-packs
by Alexey Loubyansky
Me and Stuart have been thinking about how to express feature-pack
package selection in an XML. Each one came up with a proposal but we
appear to have slightly different preferences. In case anybody has an
opinion or a better suggestion, please, share.
Brief description: feature-pack consists of packages. A package is a
unit of content. So a set of packages determines the target installation
content-wise. Feature-pack has a set of default packages. These are the
packages that get installed by default, i.e. when the user installs the
feature-pack w/o specifying any package preferences. In addition to the
default ones a feature-pack may contains non-default packages, these are
present in the feature-pack but will be installed only if the user
explicitly asks for them.
So, the question is how to express these package preferences in an XML.
Proposal 1.
- include-default flag (element or attribute) which defaults to true
(meaning the default packages will be included by default);
- if include-default is false (meaning nothing is installed by default),
then 'include' element can be used to pick the specific packages
(default and non-default ones) to be installed;
- otherwise (when include-default is true) 'exclude' element can be used
to exclude specific undesired default packages.
Proposal 2.
- exclude element - applicable to any package and means do not install
the package (and its dependencies);
- include element - applicable to non-default packages and means do
install the non-default package (and its dependencies);
- pick element - applicable to any package and means install only the
picked package(s) ignoring other default and non-default ones. pick
cannot be used in combination with exclude and include.
Basically, 'include' and 'exclude' in both proposals are practically the
same. The difference is how the picking is expressed. In the first one,
everything is explicitly excluded and then the desired ones are
explicitly included, in the second one the desired ones are simply
explicitly picked.
Thanks,
Alexey
8 years, 2 months
Remove PL from WFLY
by Pedro Igor Silva
Hi,
On the last few days I've been discussing with Tomaz about the possibility to remove PL from WFLY.
The reasons for that are:
- PicketLink was deprecated in favor of Keycloak. See [1] and [2].
- Fixes are only being done to product version by GSS/SEG teams.
- Most PL IdM and Federation capabilities such as SAML (plus a plenty of other things) are now available from Keycloak.
I think we can still provide an installer (we already have that) that could be used to enable PL to a WFLY installation. That would help people using PL to continue with their designs. But for PL Federation and Subsystem, Keycloak should be the best way to go.
Any comments ?
Regards.
Pedro Igor
[1] http://picketlink.org/news/2015/03/10/PicketLink-and-Keycloak-project-merge/
[2] http://picketlink.org/keycloak-merge-faq/
8 years, 2 months
