[jboss-as7-dev] Modularity is the spawn of Lucifer and a stinking donkey

[Scott Marlow]

On 05/21/2012 06:40 PM, Steve Ebersole wrote:
> Hey all, just to tie this up with a bow I wanted to send my take away
> from the discussions we had on IRC and make sure everyone had the same
> take away. Jason and David, wanted to thank you guys again for your time
> and input. It really helped out. I also went ahead and copied/pasted the
> transcript so we had record of discussion.
>
> The basic idea is to control this via a specific deployment descriptor
> (jboss-hibernate.xml or somesuch).

My understanding is that this jboss-hibernate.xml is an optional 
deployment descriptor that would be used by a deployment (EE or non-EE) 
that wants to use any of the following technologies:

* Hibernate Search
* Hibernate OGM
* Hibernate Integrator module implementation

The result of including a jboss-hibernate.xml, would be that the 
deployment includes the needed Hibernate modules (Hibernate 
Search/OGM/other custom modules).

If there is no jboss-hibernate.xml, there would be no change from 
current AS behaviour (OGM could be packaged with the application but 
cannot be used as a module like other persistence providers).

This in effect will allow Hibernate ORM to use the application 
deployment as a composite module that also contains the Hibernate 
Search/OGM/Other Integrator module implementation(s).

This might require some Hibernate ORM changes or not (depending on 
whether its service loader searches in the TCCL currently).  We can 
probably use the jboss-deployment-structure.xml as an alternative (short 
term solution) before we have this new Hibernate deployer that looks for 
the presence of the jboss-hibernate.xml file.  I think more discussion 
is needed about the contents of jboss-hibernate.xml before we can 
implement that.

How does that sound?

It would, in the general case,
> provide 2 pieces of information:
> 1) Which Hibernate version are we talking about. This is a very
> high-level concept of version, mainly 3 or 4.
> 2) Which "modules" do you want included. OGM? Envers? Etc. It won't name
> modules per-se in terms of AS modules. But more what Hibernate
> integrations do you want included. We will also allow for custom AS
> modules to be named here, since they could also be "Hibernate
> integrations" (provide service loader locatable impls of Integrator).
> Additionally, the inclusion of a module here will export the classes
> from the corresponding AS module into the app classloader.
>
> To account for app deployments that contain Hibernate jars, the
> suggestion was to allow a flag in this deployment descriptor to disable
> the inclusion of AS provided Hibernate modules/jars. This could also be
> leveraged to, for example, bundle an alternate JPA provider.
>
> There is still some gray area around what to do when the app deployment
> contains Hibernate jars but has no deployment descriptor. Its kind of a
> 50/50 proposition and seems like there is not a good solution.
>
>
> On Mon 21 May 2012 02:25:47 PM CDT, Scott Marlow wrote:
>> On 05/10/2012 03:04 PM, Jason T. Greene wrote:
>>> On 5/10/12 11:21 AM, David M. Lloyd wrote:
>>>> On 05/10/2012 10:24 AM, Scott Marlow wrote:
>>>>> On 05/09/2012 02:01 PM, David M. Lloyd wrote:
>>>>>> OK I admit I LOL'ed.
>>>>>>
>>>>>> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>>>>>>> Now that I have your attention, I'd like to discuss issues we are
>>>>>>> experiencing when trying to modularize the Hibernate portfolio and
>>>>>>> make it work in AS 7.1.
>>>>>>>
>>>>>>> ## Disclaimer
>>>>>>>
>>>>>>> I perfectly understand all the coolness about modularity (speed,
>>>>>>> easier dependency isolation etc). I have also carefully read :
>>>>>>>
>>>>>>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>>>>>>> - https://community.jboss.org/wiki/ModularSerialization
>>>>>>>
>>>>>>> But these tend to avoid the more complex cases of portable libraries
>>>>>>> that ought to run even outside AS 7 but have a wide variety of class
>>>>>>> and resource loading needs.
>>>>>>> I am not a complete modularity bozo but I am definitely not familiar
>>>>>>> with JBoss Modules nor similar solution.
>>>>>>>
>>>>>>> ## Requirements / Landscape
>>>>>>>
>>>>>>> Hibernate ORM uses the notion of service registry and integrator
>>>>>>> object that help during the integration or customization of the
>>>>>>> engine behavior by third-party frameworks.
>>>>>>> Enlistment of Integrators is done via the service locator pattern (a
>>>>>>> service file in META-INF/services/ that is looked up and contain the
>>>>>>> implementation class(es) at stake.
>>>>>>>
>>>>>>> Hibernate Envers is one of those customizer that depends on
>>>>>>> Hibernate
>>>>>>> ORM. Note that the core of Hibernate ORM does not depend on
>>>>>>> Hibernate
>>>>>>> Envers. The service locator file is contained in Hibernate Envers
>>>>>>> JAR.
>>>>>>> Hibernate OGM likewise, heavily customizes ORM and depends on
>>>>>>> Hibernate ORM classes - the reverse is not true. The service locator
>>>>>>> file is contained in Hibernate OGM JAR.
>>>>>>> Hibernate Search optionally depend on Hibernate ORM and JPA. The
>>>>>>> core
>>>>>>> of Hibernate Search is independent but an Hibernate Search ORM
>>>>>>> module
>>>>>>> has an integrator implementation. On top of that, Hibernate Search
>>>>>>> optionally depend on some JPA classes and behaves differently if
>>>>>>> they
>>>>>>> are there - we look them up in the classpath by reflection.
>>>>>>>
>>>>>>> On top of that, these projects do load resources (config files,
>>>>>>> classes):
>>>>>>>
>>>>>>> - from what Jason calls a Deployment classloader (the user
>>>>>>> application classes and resources really) - entities, custom
>>>>>>> analyzer
>>>>>>> implementations, resources files etc. A user could even write a
>>>>>>> custom Integrator and use the service locator pattern from his
>>>>>>> application.
>>>>>>> - from direct dependencies (Lucene is a declared dependency of
>>>>>>> Hibernate Search)
>>>>>>> - from dependencies of the deployment: for example an app developer
>>>>>>> adds the phonetic analyzer as a dependency of his application and
>>>>>>> ask
>>>>>>> Hibernate Search to use it
>>>>>>> - from modules that use these projects. Modeshape and Capedwarf are
>>>>>>> being modularized and are making use of Hibernate Search as a
>>>>>>> module.
>>>>>>> Properly loading the necessary classes located in Modeshape or
>>>>>>> Capedwarf's module but from Hibernate Search's engine proves to be
>>>>>>> very hard in our current approach.
>>>>>>>
>>>>>>> All of these projects should be able to run outside JBoss AS 7, so a
>>>>>>> modular friendly solution should somehow be abstracted and generic
>>>>>>> enough.
>>>>>>>
>>>>>>> ## What solution?
>>>>>>>
>>>>>>> More and more projects are being modularized including ones with
>>>>>>> complex resource loading dependencies like the ones I have
>>>>>>> described.
>>>>>>> AFAIK Infinispan is even in a worse situation as clustering and late
>>>>>>> class binding is at stake but let's put this one aside.
>>>>>>> I'd love to get a reference design outcome from this thread that
>>>>>>> could be copied across all these projects and future ones like Bean
>>>>>>> Validation.
>>>>>>>
>>>>>>> Today, we mostly use the good old and simple TCCL model which works
>>>>>>> fine if the jars are directly embedded in the app but fail the
>>>>>>> minute
>>>>>>> we start to move these dependencies into modules. Sanne, Strong,
>>>>>>> Scott Marlow and I are using dangerous amount of Advil to try and
>>>>>>> make everything work as expected. Some help would be awesome.
>>>>>>>
>>>>>>> To sum up:
>>>>>>>
>>>>>>> - can the Hibernate portfolio be supported within JBoss Module and
>>>>>>> how?
>>>>>>> - what kind of ClassloaderService contract should we use within
>>>>>>> these
>>>>>>> projects to be modular friendly (JBoss Modules and others)?
>>>>>>> - would such contract be generic enough to be extrapolated to
>>>>>>> JSRs in
>>>>>>> need of modular friendliness?
>>>>>>> - how to solve the chicken and egg issue of the bootstrapping: if we
>>>>>>> need to pass a ClassloaderService impl?
>>>>>> How do we do that best in a modular environment without forcing the
>>>>>> application developer to implement such godforsaken
>>>>>> ClassloaderService
>>>>>> contract or even worse pass directly to us the right classloader for
>>>>>> each call.
>>>>>>
>>>>>> I'll just start at the beginning and you can skip over the
>>>>>> background if
>>>>>> you like.
>>>>>>
>>>>>> The key starting concept is that a class' (or package's) identity is
>>>>>> not
>>>>>> just its name but also its class loader. This is the underlying
>>>>>> (existing) truth that modularity brings to the fore. Corollary to
>>>>>> this
>>>>>> are the fact that a single VM may have more than one class or package
>>>>>> with the same name, as well as the fact that not all classes/packages
>>>>>> are always directly visible from a given class loader.
>>>>>>
>>>>>> This problem (as you've seen) manifests itself primarily when you're
>>>>>> locating a class or a resource by name. You basically have two
>>>>>> options.
>>>>>> You can search *relative* to a class loader (most commonly, TCCL,
>>>>>> though using a single explicit class loader or the caller's class
>>>>>> loader
>>>>>> also fall into this category). Or, you can use the *absolute*
>>>>>> identity
>>>>>> of a class.
>>>>>>
>>>>>> Using relative resolution is often a perfectly adequate solution when
>>>>>> you're loading a single class or resource; in fact for some cases
>>>>>> (like
>>>>>> ServiceLoader for example) it's a perfect fit in conjunction with
>>>>>> TCCL
>>>>>> (in its capacity as an identifier for the "current" application). You
>>>>>> want the user to be able to specify their implementation of
>>>>>> something,
>>>>>> and you want it to be reasonably transparent; ServiceLoader+TCCL does
>>>>>> this fairly well.
>>>>>>
>>>>>> ServiceLoader also does well from the perspective of APIs with a
>>>>>> static,
>>>>>> fixed number of implementations. In this case, it is appropriate
>>>>>> for a
>>>>>> framework to have ServiceLoader use the class loader of the framework
>>>>>> itself. The framework would then be sure to import the
>>>>>> implementations
>>>>>> in question (including their service descriptors); in our modular
>>>>>> environment, which we call a "service import". Note that this often
>>>>>> means there is a circular dependency between API and implementation:
>>>>>> that's OK!
>>>>>
>>>>> We currently use this for envers but that doesn't seem as desirable
>>>>> for
>>>>> other members of the Hibernate portfolio that may be on a separate
>>>>> lifecycle. For example, the Hibernate OGM is a persistence provider
>>>>> that
>>>>> depends on Hibernate ORM. If we have Hibernate ORM depend on OGM, that
>>>>> limits the number of OGM versions that can be in use on AS7.
>>>>
>>>> Cases where implementations are pluggable could be handled using
>>>> TCCL or
>>>> by specifying a class loader, or by explicit registration at run time.
>>>> As I said above, you would only use static dependencies if the
>>>> implementations are more or less fixed.
>>>>
>>>>> Would it be possible, to add a MSC enhancement, that allows an inverse
>>>>> dependency service loader dependency to be expressed? Such that it
>>>>> would
>>>>> be enough to only have OGM depend on ORM (with an inverse service
>>>>> dependency specified). I'm thinking that the OGM module would need to
>>>>> exchange the service dependency information with the ORM module and
>>>>> clear it, when OGM goes away.
>>>>
>>>> MSC has absolutely nothing to do with any of this. Module loading is
>>>> static and is not affected by services in any way.
>>>>
>>>> In your case you probably want a registration system. But be aware that
>>>> if you have more than one OGM implementation, you're not going to be
>>>> able to use the ORM API solely to load them anyway (how would you tell
>>>> it which one you want?); instead you'd need some mechanism to specify,
>>>> so why not use TCCL or explicit class loader specification, as
>>>> appropriate?
>>>
>>> In case it's helpful. The way he handle JAXP is that we have a container
>>> default and we also support TCCL.
>>>
>>> The container default is defined by a special module:
>>> https://github.com/jbossas/jboss-as/blob/master/build/src/main/resources/modules/javax/xml/jaxp-provider/main/module.xml
>>>
>>>
>>>
>>>
>>>
>>> The rules are
>>>
>>> 1. Use service loader on TCCL (users deployment first), if an impl is
>>> found use that
>>> 2. If not result is find use the default implementation, loaded from the
>>> classloader of the "jaxp-provider" default module.
>>> 3. If the default jaxp-provider module doesn't contain it, we use the
>>> JVM default, whatever it may be.
>>>
>>> The way we implement this relies on installing a false, delegating JAXP
>>> provider. Although in our own code we should define SPIs of some sort.
>>>
>>> So for example, Hibernate OGM could provide a ProviderLocator interface,
>>> which AS would then implement. Our implementation would follow similar
>>> logic as above, although maybe you want slightly different rules. OGM
>>> would ship with a predefined ProviderLocator which would simply search
>>> TCCL.
>>>
>>
>> David/Jason,
>>
>> It might be helpful if we discussed the Hibernate ORM case on IRC
>> (either in #Hibernate-dev or #jboss-as7 (or a private room perhaps).
>> Do you guys have any time this week for that?
>>
>> Scott
>
> --
> steve at hibernate.org
> http://hibernate.org