[forge-dev] shading!?
Lincoln Baxter, III
lincolnbaxter at gmail.com
Wed Apr 20 13:12:24 EDT 2011
Actually, if we can still use the Maven loading strategy with JBoss Modules,
I think that's the best approach for the developer. Would we even need a
module descriptor in that case, or can we overload that part of the system
to basically accept what we give it as a "module" with no dependencies (that
we also provide via maven aether)?
So in a sense, gain classloader isolation while providing our own dependency
management.
I think it's time for me to dive into JBoss modules now, why oh why did I
have to present at JBW??? :)
~Lincoln
On Wed, Apr 20, 2011 at 1:07 PM, Lincoln Baxter, III <
lincolnbaxter at gmail.com> wrote:
> David,
>
> If we use a blank/empty modules descriptor, will JARs in plugin.jar:/lib/*
> be loaded and isolated from the rest of the system? If so, that might be
> exactly what we need, if combined with an appopriate maven packaging
> strategy at build time.
>
> ~Lincoln
>
>
> On Wed, Apr 20, 2011 at 12:34 PM, Lincoln Baxter, III <
> lincolnbaxter at gmail.com> wrote:
>
>>
>>
>> I think what JBoss Modules can give you is a functional, safe
>>> multiple-delegate class loader which you can't get with URL class loaders.
>>> Specifically, f you want to have a _graph_ of class loaders (as opposed to
>>> individual flat, merged classloaders) then you need something like Modules.
>>>
>>
>> I think I just embarrassed myself ;) But can weld even handle this kind of
>> separate class-loader situation?
>>
>>
>>> As far as Maven integration, you could write a ModuleLoader which does
>>> this for you. You'd have to do version management on your own though.
>>> Perhaps you could use Aether for the purpose of resolving and fetching
>>> artifacts, and then use information exposed therefrom to construct the
>>> module graph.
>>>
>>
>> Right, which is something Forge can already do pretty handily (using
>> Aether):
>> https://github.com/seam/forge/blob/master/shell-api/src/main/java/org/jboss/seam/forge/project/dependencies/DependencyResolverProvider.java
>>
>>
>>> http://www.sonatype.com/people/2010/08/introducing-aether/
>>>
>>> Your application does not have to be modular to use JBoss Modules. Though
>>> of course in my opinion that makes it much nicer. :-)
>>>
>>>
>>> On 04/20/2011 10:56 AM, Lincoln Baxter, III wrote:
>>>
>>>> I agree with both of you guys, which is why I am soliciting ideas for
>>>> this now.
>>>>
>>>> I can also say that we do not have one big Classpath right now (unless I
>>>> am horribly mistaken.) Forge already supports classpath isolation of
>>>> individual plugins, so we're already half-way there. If my understanding
>>>> of classloaders is at all right, It would not be too much work to
>>>> implement a maven resolver and additional level of Composite
>>>> classloaders to achieve this goal.
>>>>
>>>> I don't believe we need to pull in a heavyweight module system like
>>>> OSGI, or force developers to do even the simple configuration for
>>>> something like JBoss modules (which afaik, does not have a way of
>>>> packaging the actual dependencies with the module, or calling out to
>>>> Maven for dependencies - which forge can already do.)
>>>>
>>>> *This is what we have right now:* (see today.png)
>>>>
>>>> Notice how the Plugin ClassLoaders may reference Forge APIs, but the
>>>> plugins may not access each other's classloaders because there is no
>>>> link from Forge APIs to the Plugins.
>>>>
>>>> *And this is all I think it should take to do what I am suggesting:*
>>>> (see tomorrow.png)
>>>>
>>>> If it is technically possible to achieve with JBoss modules, via adding
>>>> in parsing the POM and including dependencies automatically, then I am
>>>> all for it, but from what I understand, this would not really be a 1-1
>>>> mapping and will not be trivial. I know my understanding of ClassLoaders
>>>> may be a bit Naive, but I don't think that we need to use JBoss modules
>>>> (or OSGi) to achieve this goal of automatic downloading and loading of
>>>> plugin dependency JARs.
>>>>
>>>> We certainly do not need the ability (yet) to drop an individual module
>>>> while Forge is running, because Weld does not support this kind of
>>>> disruption in the BeanStore. If you remove a JAR while Weld is running,
>>>> you get a big fiery explosion. Same for loading a single JAR, you have
>>>> to restart Weld because it will not be aware of the changes, and there
>>>> is not a good API to make it so. It still has to build the bean graph
>>>> all over again. That is one of the big reasons to use a fully-fledged
>>>> module system, and we don't have that requirement. Forge simply dumps
>>>> all classloaders (until JDK 7 we can't close them easily, yeah memory
>>>> leaks, whatever, not really important right now,) then builds a new
>>>> network of ClassLoaders and starts over.
>>>>
>>>> However, if we can extend JBoss modules with Maven support in this way,
>>>> and it's something that David would be interested in, then I think
>>>> that's one big reason to do it that way. Until then, if we can do it
>>>> quickly my way, I'd like to get it working sooner rather than later.
>>>> Please please tell me (I'm sure I don't even have to ask) if there are
>>>> gaping technical holes in what I am proposing, but if there are not,
>>>> then I'm going to stick to this position until someone bursts my bubble.
>>>>
>>>> ~Lincoln
>>>>
>>>>
>>>> On Wed, Apr 20, 2011 at 9:33 AM, Paul Bakker <paul.bakker.nl
>>>> <http://paul.bakker.nl>@gmail.com <http://gmail.com>> wrote:
>>>>
>>>> Yeah, I'm afraid that's right. I think the module system integration
>>>> should get very high priority before we reach a 1.0 version. It's
>>>> something that affects the whole way the core is built up, and it
>>>> will be very difficult to change that later on.
>>>>
>>>> Paul
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Apr 20, 2011 at 3:17 PM, Max Rydahl Andersen
>>>> <max.andersen at redhat.com <mailto:max.andersen at redhat.com>> wrote:
>>>>
>>>> +1
>>>>
>>>> I agree with what you want to do Lincoln, but don't see how that
>>>> can be made to happen without a module system.
>>>>
>>>> Until then everything just will have to live in one big
>>>> classpath.
>>>>
>>>> /max
>>>>
>>>> On Apr 20, 2011, at 14:24, Paul Bakker wrote:
>>>>
>>>> > And that's why a module system is needed. If people can just
>>>> add dependencies there will be duplicate (possibly multiple
>>>> versions) dependencies. Each plugin should be in a separate
>>>> classloader, and each dependency should be too so that a plugin
>>>> never breaks other plugins or Forge itself. It would be a big
>>>> limitation if plugins can't use libraries otherwise.
>>>> >
>>>> > Paul
>>>> >
>>>> > On Tue, Apr 19, 2011 at 6:14 PM, Lincoln Baxter, III
>>>> <lincolnbaxter at gmail.com <mailto:lincolnbaxter at gmail.com>>
>>>> wrote:
>>>> > That's right, but what I'm saying is that I don't want
>>>> developers to be responsible for anything but *their code* -- if
>>>> they have dependencies, those depenencies will be fetched for
>>>> them (or somehow bundled in the JAR file itself, which is
>>>> certainly possible, however not my preference.)
>>>> >
>>>> > If you could write a plugin, reference dependencies in your
>>>> POM, and have everything *Just Work* don't you think that would
>>>> be a friendlier experience?
>>>> >
>>>> > ~Lincoln
>>>> >
>>>> >
>>>> > On Mon, Apr 18, 2011 at 5:55 PM, Max Andersen
>>>> <manderse at redhat.com <mailto:manderse at redhat.com>> wrote:
>>>> > But if your plugin uses multiple jars it is not one jar.
>>>> >
>>>> >
>>>> > /max (sent from my phone)
>>>> >
>>>> >
>>>> > On 18/04/2011, at 19.06, "Lincoln Baxter, III"
>>>> <lincolnbaxter at gmail.com <mailto:lincolnbaxter at gmail.com>>
>>>> wrote:
>>>> >
>>>> >> That doesn't solve the problem of having to drop jar files
>>>> onto the classpath in order for plugins to work. I want one JAR
>>>> per plugin.
>>>> >>
>>>> >> ~Lincoln
>>>> >>
>>>> >> On Mon, Apr 18, 2011 at 12:59 PM, Max Rydahl Andersen
>>>> <max.andersen at redhat.com <mailto:max.andersen at redhat.com>>
>>>> wrote:
>>>> >>
>>>> >> On Apr 18, 2011, at 18:57, Lincoln Baxter, III wrote:
>>>> >>
>>>> >> > I don't want to force plugin-developers to create modules
>>>> for every dependency that their plugin requires. That's why I've
>>>> been avoiding OSGI or JBoss Modules.
>>>> >>
>>>> >> But then you shouldn't be forcing them to shade either - you
>>>> should just have one global classloader for the plugins then.
>>>> >>
>>>> >> /max
>>>> >>
>>>> >> >
>>>> >> > ~Lincoln
>>>> >> >
>>>> >> > On Mon, Apr 18, 2011 at 12:37 PM, Max Rydahl Andersen
>>>> <max.andersen at redhat.com <mailto:max.andersen at redhat.com>>
>>>> wrote:
>>>> >> >
>>>> >> > > What do you think about using Maven APIs to inspect the
>>>> POM and fetch dependencies dynamically for each plugin, then
>>>> isolate them in the plugin's classloader?
>>>> >> >
>>>> >> > Why not just load them in to one classloader so you don't
>>>> have collisions when there are mixed dependencies on Forge it
>>>> self ?
>>>> >> >
>>>> >> > How about shared data instances ? How does that work ?
>>>> >> >
>>>> >> > ...as a side note...creating our own module system now - I
>>>> feel that is a very bad direction :(
>>>> >> > Might as well adopt osgi plugin system if you want this
>>>> kind of separation ?
>>>> >> >
>>>> >> > /max
>>>> >> >
>>>> >> > >
>>>> >> > > ~Lincoln
>>>> >> > >
>>>> >> > > On Mon, Apr 18, 2011 at 11:56 AM, Max Rydahl Andersen
>>>> <max.andersen at redhat.com <mailto:max.andersen at redhat.com>>
>>>> wrote:
>>>> >> > >
>>>> >> > > On Apr 18, 2011, at 15:15, Lincoln Baxter, III wrote:
>>>> >> > >
>>>> >> > > > "if there is a standard location for dependencies"
>>>> >> > > >
>>>> >> > > > What do you mean?
>>>> >> > >
>>>> >> > > Your "standard" for shading is that you put all classes
>>>> into the plugin.jar.
>>>> >> > >
>>>> >> > > A "standard" for dependencies for a plugin.jar could be
>>>> "next to the plugin.jar".
>>>> >> > >
>>>> >> > > Would still have the problem of overlapping jars but
>>>> then at least its easier to see where the duplication is.
>>>> >> > >
>>>> >> > > /max
>>>> >> > >
>>>> >> > > >
>>>> >> > > > Thx,
>>>> >> > > > ~Lincoln
>>>> >> > > >
>>>> >> > > > On Mon, Apr 18, 2011 at 6:20 AM, Max Rydahl Andersen
>>>> <max.andersen at redhat.com <mailto:max.andersen at redhat.com>>
>>>> wrote:
>>>> >> > > >
>>>> >> > > > > I was thinking we might already be able to do that
>>>> using the existing pom.xml metadata that's stored in the
>>>> artifact itself, or is that too tricky?
>>>> >> > > >
>>>> >> > > > if there is a standard location for dependencies then
>>>> it should be fine - at least better than shading ;)
>>>> >> > > >
>>>> >> > > > /max
>>>> >> > > >
>>>> >> > > > >
>>>> >> > > > > On Mon, Apr 11, 2011 at 6:51 PM, Max Andersen
>>>> <manderse at redhat.com <mailto:manderse at redhat.com>> wrote:
>>>> >> > > > > I was thinking Plugin jar having references to
>>>> dependent jars via manifest.mf
>>>> >> > > > >
>>>> >> > > > > /max (sent from my phone)
>>>> >> > > > >
>>>> >> > > > >
>>>> >> > > > > On 12/04/2011, at 00.39, "Lincoln Baxter, III"
>>>> <lincolnbaxter at gmail.com <mailto:lincolnbaxter at gmail.com>>
>>>> wrote:
>>>> >> > > > >
>>>> >> > > > >> Can you give an example of how you would bundle the
>>>> JARs? (Just put them in /META-INF/dependencies/ ... ?) And would
>>>> that not cause just as many class conflicts? If you
>>>> shade/relocate then the deps *should be* completely isolated.
>>>> >> > > > >>
>>>> >> > > > >> On Mon, Apr 11, 2011 at 4:11 PM, Max Rydahl
>>>> Andersen <max.andersen at redhat.com
>>>> <mailto:max.andersen at redhat.com>> wrote:
>>>> >> > > > >> well, recommending just bundling jars would be a
>>>> better approach than shading IMO.
>>>> >> > > > >>
>>>> >> > > > >> /max
>>>> >> > > > >>
>>>> >> > > > >> On Apr 11, 2011, at 16:00, Lincoln Baxter, III
>>>> wrote:
>>>> >> > > > >>
>>>> >> > > > >> > Yeah, shading is currently the recommended
>>>> approach. Conflicts should be avoided by using relocations. I
>>>> know this is... not a great method, but for now it's all we've
>>>> got. Open to suggestions.
>>>> >> > > > >> >
>>>> >> > > > >> > ~Lincoln
>>>> >> > > > >> >
>>>> >> > > > >> > On Mon, Apr 11, 2011 at 3:41 AM, Max Rydahl
>>>> Andersen <max.andersen at redhat.com
>>>> <mailto:max.andersen at redhat.com>> wrote:
>>>> >> > > > >> > Heya,
>>>> >> > > > >> >
>>>> >> > > > >> > Lincoln, I just saw your commits to hibernattools
>>>> plugin at
>>>> (
>>>> https://github.com/forge/plugin-hibernate-tools/commit/8b208b4a8e79dbb8a01d10d266ee81afd2cf7106
>>>> )
>>>> >> > > > >> >
>>>> >> > > > >> > Is shading of jars really the recommended
>>>> approach for plugins in Forge ?
>>>> >> > > > >> >
>>>> >> > > > >> > How are you going to share/avoid collisions of
>>>> libraries across plugins if they need to bundle via shading ?
>>>> >> > > > >> >
>>>> >> > > > >> > /max
>>>> >> > > > >> > http://about.me/maxandersen
>>>> >> > > > >> >
>>>> >> > > > >> >
>>>> >> > > > >> >
>>>> >> > > > >> >
>>>> >> > > > >> > _______________________________________________
>>>> >> > > > >> > forge-dev mailing list
>>>> >> > > > >> > forge-dev at lists.jboss.org
>>>> <mailto:forge-dev at lists.jboss.org>
>>>>
>>>> >> > > > >> >
>>>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>>> >> > > > >> >
>>>> >> > > > >> >
>>>> >> > > > >> >
>>>> >> > > > >> > --
>>>> >> > > > >> > Lincoln Baxter, III
>>>> >> > > > >> > http://ocpsoft.com
>>>> >> > > > >> > http://scrumshark.com
>>>> >> > > > >> > "Keep it Simple"
>>>> >> > > > >>
>>>> >> > > > >> /max
>>>> >> > > > >> http://about.me/maxandersen
>>>> >> > > > >>
>>>> >> > > > >>
>>>> >> > > > >>
>>>> >> > > > >>
>>>> >> > > > >>
>>>> >> > > > >>
>>>> >> > > > >> --
>>>> >> > > > >> Lincoln Baxter, III
>>>> >> > > > >> http://ocpsoft.com
>>>> >> > > > >> http://scrumshark.com
>>>> >> > > > >> "Keep it Simple"
>>>> >> > > > >
>>>> >> > > > >
>>>> >> > > > >
>>>> >> > > > > --
>>>> >> > > > > Lincoln Baxter, III
>>>> >> > > > > http://ocpsoft.com
>>>> >> > > > > http://scrumshark.com
>>>> >> > > > > "Keep it Simple"
>>>> >> > > >
>>>> >> > > > /max
>>>> >> > > > http://about.me/maxandersen
>>>> >> > > >
>>>> >> > > >
>>>> >> > > >
>>>> >> > > >
>>>> >> > > >
>>>> >> > > >
>>>> >> > > > --
>>>> >> > > > Lincoln Baxter, III
>>>> >> > > > http://ocpsoft.com
>>>> >> > > > http://scrumshark.com
>>>> >> > > > "Keep it Simple"
>>>> >> > >
>>>> >> > > /max
>>>> >> > > http://about.me/maxandersen
>>>> >> > >
>>>> >> > >
>>>> >> > >
>>>> >> > >
>>>> >> > >
>>>> >> > >
>>>> >> > > --
>>>> >> > > Lincoln Baxter, III
>>>> >> > > http://ocpsoft.com
>>>> >> > > http://scrumshark.com
>>>> >> > > "Keep it Simple"
>>>> >> >
>>>> >> > /max
>>>> >> > http://about.me/maxandersen
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > --
>>>> >> > Lincoln Baxter, III
>>>> >> > http://ocpsoft.com
>>>> >> > http://scrumshark.com
>>>> >> > "Keep it Simple"
>>>> >>
>>>> >> /max
>>>> >> http://about.me/maxandersen
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Lincoln Baxter, III
>>>> >> http://ocpsoft.com
>>>> >> http://scrumshark.com
>>>> >> "Keep it Simple"
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Lincoln Baxter, III
>>>> > http://ocpsoft.com
>>>> > http://scrumshark.com
>>>> > "Keep it Simple"
>>>> >
>>>> > _______________________________________________
>>>> > forge-dev mailing list
>>>> > forge-dev at lists.jboss.org <mailto:forge-dev at lists.jboss.org>
>>>>
>>>> > https://lists.jboss.org/mailman/listinfo/forge-dev
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > forge-dev mailing list
>>>> > forge-dev at lists.jboss.org <mailto:forge-dev at lists.jboss.org>
>>>>
>>>> > https://lists.jboss.org/mailman/listinfo/forge-dev
>>>>
>>>> /max
>>>> http://about.me/maxandersen
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> forge-dev mailing list
>>>> forge-dev at lists.jboss.org <mailto:forge-dev at lists.jboss.org>
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> forge-dev mailing list
>>>> forge-dev at lists.jboss.org <mailto:forge-dev at lists.jboss.org>
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Lincoln Baxter, III
>>>> http://ocpsoft.com
>>>> http://scrumshark.com
>>>> "Keep it Simple"
>>>>
>>>
>>>
>>> --
>>> - DML
>>>
>>
>>
>>
>> --
>> Lincoln Baxter, III
>> http://ocpsoft.com
>> http://scrumshark.com
>> "Keep it Simple"
>>
>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.com
> http://scrumshark.com
> "Keep it Simple"
>
--
Lincoln Baxter, III
http://ocpsoft.com
http://scrumshark.com
"Keep it Simple"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/forge-dev/attachments/20110420/41fac13d/attachment-0001.html
More information about the forge-dev
mailing list