[hibernate-dev] [jdf-dev] JBoss Modules dependency vs pain POM dependency + exclusions

[Sanne Grinovero]

Great points. Let me try attack some inline :-)
I'm also adding Brian and Jason in CC in hope they have better
thoughts and are willing to correct me.

On 7 June 2013 18:25, Hardy Ferentschik <hardy at hibernate.org> wrote:
>
> On 7 Jan 2013, at 5:44 PM, Sanne Grinovero <sanne at hibernate.org> wrote:
>
>> On 7 June 2013 16:12, Hardy Ferentschik <hardy at hibernate.org> wrote:
>>> Again, late to the party, but I agree with Emmanuel. Modules is imo to AS specific in order to encourage it
>>> as a default/referred configuration approach. I see it more as an expert mode to configure your app server.
>>
>> Right we should make it easier and with clear documentation. But
>> "dropping something in your /modules" isn't sooo different than
>> dropping an EAR in your /deploy
>
> Yes it is and as Gunnar pointed out there might also be more resistance from the ops people to to something like this

Those are very valid concerns. I've personally seen a variety of
policies, ranging from "let's replace the JBoss jars with latest
Hibernate snapshot" (done that myself in production)
to "there's a testing process of 6 months for each configuration
change you want on the AS; the overall cost of the tests is 2 million,
so the first step is to file a business case to justify that" (at
which point I've quit the employer - true story).
But generally it used to boil down to "who is responsible" and "what
does the documentation recommend". Which in turn boils down to what we
can test, and what we write in the documentation.

But if you look at the modules directory in WildFly, the "traditional"
jars are bundled in a directory starting with:
/modules/system/layers/base/

There is an explanation about this structure here:
https://community.jboss.org/wiki/LayeredDistributionsAndModulePathOrganization

I couldn't find a full explanation of what we're expected to see in
/modules other than "system", or in layers other than "base" but it
looks like quite self-explanatory that at least power users are
expected to drop in additional modules. If you get our libraries via a
package manager, that might get very easy and especially if we bless
the process by testing for this configuration and recommend it in the
documentation, I think ops people could be willing to accept it - I
guess there is some kind of parallelism historically with the *-DS.xml
datasource files to be deployed?

I think it's clear that layers are expected to provide such modules,
and the Hibernate Search project specifically should provide these for
other projects to consume. You're right that this doesn't need to be
"the" recommended way for Hibernate Search / ORM users but there are
at least some benefits, mostly of monitoring and administration
capabilities, which could make ops people specifically less resistant
about the new approach.

>> both are easily scripted, we have a not-too-nice solution which works
>> with Maven, and if it becomes a mainstream strategy for other JBoss
>> modules then we could certainly make a better looking Maven plugin.
>
> See, now we start talking. If we have the right tools we can start talking about this.
> The current scripts are more of a hack than anything and I would not like people to
> do the same thing. Also, what is this maven plugin supposed to do? Patch your local
> AS instance. That's fine for your local build, but how does this help production?
> What are you going to deliver to the guys running the actual production app? An ear file
> and a zip with the instructions to please unzip this first here?

Depends on the system. Red Hat users might "just have it there"
automatically provided because it's actived on their subscription
account, so the RPM automatically includes it in the application
server; this has no downsides besides some disk space as it would only
get activated by deployments specifying the need.
Fedora users could "yum install" it, while developer workstations
could get it bundled with other tooling such as Developer Studio. Some
might need to unzip it, and while I agree that's annoying it's not
different than unzipping WildFly in the first place, and trivial to
specify in any build tool, even more as we deploy the zip as Maven
modules.
Maybe we could even have a light WildFly download, and a "WildFly with
extensions".

>>> And is not also dangerous to fiddle too much with the modules? Adding new modules might be ok, but often
>>> you end up updating others as well (talking EL and Bean Validation) and there you don't know whether you
>>> break another subsystem.
>>
>> What fiddling? we don't update EL nor Bean Validation modules, but we
>> provide solid modules which are tested by our CI and consumed by other
>> projects (getting more tests done to benefit all).
>
> I am not talking Search in this case. It was a general comment against modifying
> modules directories.

Right, modifying the provided modules dir doesn't feel good.
I hope the new WildFly modules directory structure addresses this concern?

>> Getting more pratical: explain me how I should configure the classpath
>> for an "embedded lib style" deployment which takes advantage of 2nd
>> level caching via Infinispan, uses Hibernate Search to store indexes,
>> and opens a JGroups backend. The dump solution is to include both
>> Infinispan and JGroups; you'll have to add all dependencies like
>> JBLogging, JBMarshaller, a provided JTA manager.. you'll get an
>> app-server class of services but all management extensions will not be
>> wired up to the AS core, nor Infinispan will be connected to internals
>> like mod_cluster & al. I'm not even sure if the logger will bind
>> correctly for dynamic configuration or if it's going to work via a
>> Log4J bridge which gets intercepted and re-routed to the AS proper
>> logger .. lots of problems I'd not prepared to face, I bet big time
>> Tomcat would sound more appealing: no metrics (either can we), no
>> admin (wouldn't work either), but at least these services don't fight
>> against each other's resources.
>
> Sure, there might be situations where the module approach is better. I was just saying
> it should not be the "default way of doing things". I trust you that the scenarios
> you describe are best solved the way you say, but I still think it is a little unfortunate.

I'm not sure it is the best way. It seems it would solve a lot of
problems so I hope we could go to the bottom of this and verify: the
ideal solution still needs some changes, for example how the app
server can share a partial JGroups stack so that we can actually reuse
it and stay independent (I'll be exploring this with Bela and Paul
next week). Most other benefits however seem to be reachable for
Search users by using modules today.

BTW I just figured out the example I made up above was seriously evil,
as Hibernate would need to interact both with the internal application
server provided Infinispan and the one included in the application
libs for index clustering "via" the Search bridge which deals . So
it's not even about "which approach is better": I don't think it's at
all possible without deploying custom modules, or without fully
deactivating the JPA deployer.

>>> Also it adds additional constrains when deploying the app server and app. It is not just about dropping in a ear/war
>>> anymore, suddenly I have to "patch" my app server first.
>>
>> Right I see your point on this: if it feels like patching it doesn't
>> look good. But it feels much better if you look at it as "dependency
>> resolution".
>
> How can it feels better by just naming it something else. What counts is that I have to
> install a default AS instance and then modify/patch its inner configuration.

We always do modify the AS server when deploying an app or configuring
it in any form, still I never tell the top manager we're going to
initiate an open hearth surgery on their IT system :D
Such a terminology would only appropriate in some quite specific
contexts right? So let's not pick the wrong name: I don't think that
adding a jar to the place it is designed to be, in an appropriate
extensions directory, is "patching" nor has much to do with inner
configuration.

>> I can totally see how a wildfly instance could download
>> these on-demand from the dependency definition; by having these in
>> Maven, corporate environments might not dislike it too much as they
>> could have their own repository managers.
>
> Now that is different imo. Now we provide some proper tooling around this.
> When are you having this ready to go?

I think we could ask for help to make better tooling, if we have a
case. We won't get tooling to deploy modules if we don't make the
modules and verify this is a good plan. Ideally I'd like WildFly to
automatically download extensions, so we can blame the app server for
"downloading the internet" :)

>> BTW this problem is only for JBoss / Wildfly as other app servers
>> don't bundle Hibernate, so the solution is special purpose as well.
>
> So basically we are saying on our app server it is actually harder to get this to work than on
> others. Interesting. Maybe we should target Glassfish then.

We should target them all, including WildFly even if it needs an
additional zip :-)
BTW only WildFly would be able to address the monitoring, Infinispan
and mod_cluster integrations appropriately, at least until others
won't expose similar capabilities and modularity.

-- Sanne

>
> --Hardy
>