[wildfly-dev] Subsystems to advertise implicit and optional dependencies to galleon

[Brian Stansberry]

On Mon, Nov 12, 2018 at 7:28 AM, Jean-Francois Denise <jdenise at redhat.com>
wrote:

>
>
> On 09/11/2018 18:32, Brian Stansberry wrote:
>
>
>
> On Fri, Nov 9, 2018 at 2:30 AM, Jean-Francois Denise <jdenise at redhat.com>
> wrote:
>
>> Hi,
>>
>> in the galleon project, in a context where we would not provision a
>> complete server but just a subset required to run a given configuration,
>> we have identified a need for subsystems to advertise to the galleon
>> tooling some modules in addition to the modules galleon discover by
>> traversing the module dependency tree.
>>
>> The first case is DeploymentProcessor injecting modules into the
>> deployment units (implicit modules).
>>
>> The deployment injected ones are not required to be a dependency of the
>> subsystem module, so galleon has the risk to miss some of them.
>>
>> As an example, in logging subsystem we have the following non optional
>> injected modules :
>> "org.jboss.logging",
>> "org.apache.commons.logging",
>> "org.apache.log4j",
>> "org.slf4j",
>> "org.jboss.logging.jul-to-slf4j-stub"
>>
>> Some of these modules are direct dependencies of logging subsystem, some
>> others are indirect dependencies, others could be not present at all in
>> the module dependency tree (eg: org.jboss.logging.jul-to-slf4j-stub).
>>
>> So we are thinking at solving this issue in 2 possible ways:
>>
>> 1) Mandate that all injected modules become dependencies of the
>> subsystem module at the cost to load some useless modules at runtime.
>>
>> 2) Possibly better, make the subsystem to call
>> RuntimeCapability.addAdditionalRequiredPackages (package name being
>> module name) for each injected module. An existing capability or a new
>> one  would have to be created.
>>
>> There is also the case of optional injected module dependencies (eg: ee
>> subsystem org.glassfish.javax.el, org.eclipse.yasson, ...). We need to
>> treat them differently. When one is provisioning a server using galleon
>> he can choose to exclude some packages from the provisioned server.
>> Optional packages can be excluded without making the provisioned state
>> invalid (as opposed to required package that can't be excluded). These
>> optional implicit dependencies are typical usage of this, they are not
>> required by the deployment unit to properly operate.
>>
>> For these, we plan to use
>> RuntimeCapability.addAdditionalOptionalPackages. We can't make them
>> "optional" in JBoss module. When galleon provision a subset of the
>> server we don't include all optional dependencies.
>>
>> This brings the case of provisioning of optional dependencies present in
>> JBOSS module.xml.
>>
>> We have identified multiple kind of optional dependencies.
>>
>> 1) The optional dependencies that are referencing modules only in use
>> when a feature is present in the configuration of the subsystem (eg: jmx
>> subsystem optional dep on org.jboss.remoting-jmx due to
>> <remoting-connector/>, remoting subsystem optional dep on
>> io.undertow.core for <http-connector/>, elytron subsystem optional dep
>> on org.picketbox for <jacc-policy>)
>>
>> In order to have these dependencies to be provisioned with the
>> subsystem, we can attach thanks to
>> RuntimeCapability.addAdditionalRequiredPackages the modules to the
>> feature. When the feature is present in the configuration, the module is
>> no more optional but required.
>>
>> 2) The optional dependencies that reference modules that are part of
>> another subsystems and only use if this other subsystem is present (eg:
>> org.jboss.as.jpa optional dep on org.jboss.as.ejb3,
>> org.jboss.as.transactions optional dep on org.jboss.remoting). These
>> ones are simply not taken into account
>>
>> 3)  The optional dependencies that reference modules that are not part
>> of another subsystem (so are not provisioned by another source) but are
>> only meaningful if the other subsystem is present. We call these ones
>> "passive" (eg: org.jboss.as.weld optional dependency on
>> org.jboss.as.weld.ejb|jpa|beanvalidation|webservices|transactions).
>> "passive" are analyzed. If all their dependencies are present, then they
>> are included. Some implicit optional dependencies can fall into this
>> category (eg: org.jboss.jaxrs optional dep on
>> org.jboss.resteasy.resteasy-validator-provider-11).
>>
>> The passive optional dependencies would be advertised with
>> RuntimeCapability.addAdditionalPassiveOptionalPackages(optional
>> dependency package name).
>>
>> So to summarize:
>>
>> - RuntimeCapability.addAdditionalRequiredPackages for all required
>> implicit modules
>>
>> - RuntimeCapability.addAdditionalRequiredPackages to associate optional
>> dependencies to a feature
>>
>> - RuntimeCapability.addAdditionalOptionalPackages for all optional
>> implicit dependencies that are not passive
>>
>> - RuntimeCapability.addAdditionalPassiveOptionalPackages for all
>> optional dependencies (implicit or not) that are passive
>>
>
> A good question Alexey raised is whether the capability is the right
> concept for encapsulating this information.
>
> In the end the key point is that the information ends up in the galleon
> feature-spec. That's the thing that encapsulates the configuration elements
> and the packages (i.e. fs content) that are needed to provide a feature.
> It's the feature that needs the package. We generate the galleon feature
> specs from our management model information. When in WF 13 we hit the first
> must-implement use case for getting additional package info into a feature
> spec I kind of arbitrarily picked 'RuntimeCapability' as the place to
> record that. It was a reasonable enough choice for that one use case but
> isn't great otherwise.
>
> The alternative is to record this data in the ResourceDefinition, which is
> the primary source of the data that's used in the feature spec generation.
>
> The conceptual argument is a capability is all about a named provider of a
> contract that other elements of the server can rely on; e.g. some MSC
> Service with a particular value type that will be exposed.  But this
> additional package stuff isn't really about that contract, it's about other
> things that will be done that likely are not relevant to other elements of
> the system.
>
> A hint of a design smell would be if we end up creating RuntimeCapability
> instances for no other reason than to record this additional package
> information. Something I could see happening with this logging case you
> mentioned at the start.
>
> I don't think ResourceDefinition has this problem.
>
> I have introduced artificial capabilities in ee, jaxrs and weld
> (subsystems that are targeted by the demo) to add additional packages. It
> seems that it would be common case. So it is actually smelly.
>

Note though that all 3 of those subsystems are lacking capabilities for
things they provide that other subsystems use. Or perhaps in the jaxrs case
a private one that lets it require/use capabilities from other subsystems.
(Yeray is working on weld.) IOW it's possible you're not really adding
artificial capabilities, you're just bumping into the missing ones. ;)

^^^ is just an FYI comment though. I have little doubt we'll find
artificial capabilities and it's possible ee, jaxrs, weld would have them.


>
>
>
>> Is it something that subsystems owner would be ready to put in place to
>> help galleon in this task? It would require a bit of analysis of the
>> dependencies of your modules but the gain could be quite important.
>
>
> Please don't let my point above derail consideration of this question.
> Use of ResourceDefinition vs RuntimeCapability is a relatively small
> detail.
>
> Some
>> early experimentation of this has shown a big reduction of the server
>> filesystem footprint (web server + cdi has been reduced from 156MB to
>> 46MB). The runtime memory usage reduction is not that big, but less
>> modules being loaded we have a gain.
>>
>> Thanks for reading.
>>
>> JF
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
>
>
> --
> Brian Stansberry
> Manager, Senior Principal Software Engineer
> Red Hat
>
>
>


-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20181112/a0bad18d/attachment-0001.html