[forge-dev] shading!?
David M. Lloyd
dlloyd at redhat.com
Wed Apr 20 14:03:19 EDT 2011
You'd bypass the descriptor altogether and build your modules
programmatically, by specifying the dependencies and so forth. That,
and more, is what you can do with a custom module loader.
You should come to my JUDCon talks about it. :-)
On 04/20/2011 12:12 PM, Lincoln Baxter, III wrote:
> Actually, if we can still use the Maven loading strategy with JBoss
> Modules, I think that's the best approach for the developer. Would we
> even need a module descriptor in that case, or can we overload that part
> of the system to basically accept what we give it as a "module" with no
> dependencies (that we also provide via maven aether)?
>
> So in a sense, gain classloader isolation while providing our own
> dependency management.
>
> I think it's time for me to dive into JBoss modules now, why oh why did
> I have to present at JBW??? :)
>
> ~Lincoln
>
> On Wed, Apr 20, 2011 at 1:07 PM, Lincoln Baxter, III
> <lincolnbaxter at gmail.com <mailto:lincolnbaxter at gmail.com>> wrote:
>
> David,
>
> If we use a blank/empty modules descriptor, will JARs in
> plugin.jar:/lib/* be loaded and isolated from the rest of the
> system? If so, that might be exactly what we need, if combined with
> an appopriate maven packaging strategy at build time.
>
> ~Lincoln
>
>
> On Wed, Apr 20, 2011 at 12:34 PM, Lincoln Baxter, III
> <lincolnbaxter at gmail.com <mailto:lincolnbaxter at gmail.com>> wrote:
>
>
>
> I think what JBoss Modules can give you is a functional,
> safe multiple-delegate class loader which you can't get with
> URL class loaders. Specifically, f you want to have a
> _graph_ of class loaders (as opposed to individual flat,
> merged classloaders) then you need something like Modules.
>
>
> I think I just embarrassed myself ;) But can weld even handle
> this kind of separate class-loader situation?
>
> As far as Maven integration, you could write a ModuleLoader
> which does this for you. You'd have to do version
> management on your own though. Perhaps you could use Aether
> for the purpose of resolving and fetching artifacts, and
> then use information exposed therefrom to construct the
> module graph.
>
>
> Right, which is something Forge can already do pretty handily
> (using Aether):
> https://github.com/seam/forge/blob/master/shell-api/src/main/java/org/jboss/seam/forge/project/dependencies/DependencyResolverProvider.java
>
> http://www.sonatype.com/people/2010/08/introducing-aether/
>
> Your application does not have to be modular to use JBoss
> Modules. Though of course in my opinion that makes it much
> nicer. :-)
>
>
> On 04/20/2011 10:56 AM, Lincoln Baxter, III wrote:
>
> I agree with both of you guys, which is why I am
> soliciting ideas for
> this now.
>
> I can also say that we do not have one big Classpath
> right now (unless I
> am horribly mistaken.) Forge already supports classpath
> isolation of
> individual plugins, so we're already half-way there. If
> my understanding
> of classloaders is at all right, It would not be too
> much work to
> implement a maven resolver and additional level of Composite
> classloaders to achieve this goal.
>
> I don't believe we need to pull in a heavyweight module
> system like
> OSGI, or force developers to do even the simple
> configuration for
> something like JBoss modules (which afaik, does not have
> a way of
> packaging the actual dependencies with the module, or
> calling out to
> Maven for dependencies - which forge can already do.)
>
> *This is what we have right now:* (see today.png)
>
> Notice how the Plugin ClassLoaders may reference Forge
> APIs, but the
> plugins may not access each other's classloaders because
> there is no
> link from Forge APIs to the Plugins.
>
> *And this is all I think it should take to do what I am
> suggesting:*
> (see tomorrow.png)
>
> If it is technically possible to achieve with JBoss
> modules, via adding
> in parsing the POM and including dependencies
> automatically, then I am
> all for it, but from what I understand, this would not
> really be a 1-1
> mapping and will not be trivial. I know my understanding
> of ClassLoaders
> may be a bit Naive, but I don't think that we need to
> use JBoss modules
> (or OSGi) to achieve this goal of automatic downloading
> and loading of
> plugin dependency JARs.
>
> We certainly do not need the ability (yet) to drop an
> individual module
> while Forge is running, because Weld does not support
> this kind of
> disruption in the BeanStore. If you remove a JAR while
> Weld is running,
> you get a big fiery explosion. Same for loading a single
> JAR, you have
> to restart Weld because it will not be aware of the
> changes, and there
> is not a good API to make it so. It still has to build
> the bean graph
> all over again. That is one of the big reasons to use a
> fully-fledged
> module system, and we don't have that requirement. Forge
> simply dumps
> all classloaders (until JDK 7 we can't close them
> easily, yeah memory
> leaks, whatever, not really important right now,) then
> builds a new
> network of ClassLoaders and starts over.
>
> However, if we can extend JBoss modules with Maven
> support in this way,
> and it's something that David would be interested in,
> then I think
> that's one big reason to do it that way. Until then, if
> we can do it
> quickly my way, I'd like to get it working sooner rather
> than later.
> Please please tell me (I'm sure I don't even have to
> ask) if there are
> gaping technical holes in what I am proposing, but if
> there are not,
> then I'm going to stick to this position until someone
> bursts my bubble.
>
> ~Lincoln
>
>
> On Wed, Apr 20, 2011 at 9:33 AM, Paul Bakker
> <paul.bakker.nl <http://paul.bakker.nl>
> <http://paul.bakker.nl>@gmail.com <http://gmail.com>
> <http://gmail.com>> wrote:
>
> Yeah, I'm afraid that's right. I think the module
> system integration
> should get very high priority before we reach a 1.0
> version. It's
> something that affects the whole way the core is
> built up, and it
> will be very difficult to change that later on.
>
> Paul
>
>
>
>
> On Wed, Apr 20, 2011 at 3:17 PM, Max Rydahl Andersen
> <max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>
> <mailto:max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>>> wrote:
>
> +1
>
> I agree with what you want to do Lincoln, but
> don't see how that
> can be made to happen without a module system.
>
> Until then everything just will have to live in
> one big classpath.
>
> /max
>
> On Apr 20, 2011, at 14:24, Paul Bakker wrote:
>
> > And that's why a module system is needed. If people
> can just
> add dependencies there will be duplicate
> (possibly multiple
> versions) dependencies. Each plugin should be in
> a separate
> classloader, and each dependency should be too
> so that a plugin
> never breaks other plugins or Forge itself. It
> would be a big
> limitation if plugins can't use libraries otherwise.
> >
> > Paul
> >
> > On Tue, Apr 19, 2011 at 6:14 PM, Lincoln Baxter, III
> <lincolnbaxter at gmail.com
> <mailto:lincolnbaxter at gmail.com>
> <mailto:lincolnbaxter at gmail.com
> <mailto:lincolnbaxter at gmail.com>>> wrote:
> > That's right, but what I'm saying is that I don't want
> developers to be responsible for anything but
> *their code* -- if
> they have dependencies, those depenencies will
> be fetched for
> them (or somehow bundled in the JAR file itself,
> which is
> certainly possible, however not my preference.)
> >
> > If you could write a plugin, reference dependencies
> in your
> POM, and have everything *Just Work* don't you
> think that would
> be a friendlier experience?
> >
> > ~Lincoln
> >
> >
> > On Mon, Apr 18, 2011 at 5:55 PM, Max Andersen
> <manderse at redhat.com <mailto:manderse at redhat.com>
> <mailto:manderse at redhat.com
> <mailto:manderse at redhat.com>>> wrote:
> > But if your plugin uses multiple jars it is not one jar.
> >
> >
> > /max (sent from my phone)
> >
> >
> > On 18/04/2011, at 19.06, "Lincoln Baxter, III"
> <lincolnbaxter at gmail.com
> <mailto:lincolnbaxter at gmail.com>
> <mailto:lincolnbaxter at gmail.com
> <mailto:lincolnbaxter at gmail.com>>> wrote:
> >
> >> That doesn't solve the problem of having to drop jar
> files
> onto the classpath in order for plugins to work.
> I want one JAR
> per plugin.
> >>
> >> ~Lincoln
> >>
> >> On Mon, Apr 18, 2011 at 12:59 PM, Max Rydahl Andersen
> <max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>
> <mailto:max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>>> wrote:
> >>
> >> On Apr 18, 2011, at 18:57, Lincoln Baxter, III wrote:
> >>
> >> > I don't want to force plugin-developers to create
> modules
> for every dependency that their plugin requires.
> That's why I've
> been avoiding OSGI or JBoss Modules.
> >>
> >> But then you shouldn't be forcing them to shade
> either - you
> should just have one global classloader for the
> plugins then.
> >>
> >> /max
> >>
> >> >
> >> > ~Lincoln
> >> >
> >> > On Mon, Apr 18, 2011 at 12:37 PM, Max Rydahl Andersen
> <max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>
> <mailto:max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>>> wrote:
> >> >
> >> > > What do you think about using Maven APIs to
> inspect the
> POM and fetch dependencies dynamically for each
> plugin, then
> isolate them in the plugin's classloader?
> >> >
> >> > Why not just load them in to one classloader so
> you don't
> have collisions when there are mixed
> dependencies on Forge it self ?
> >> >
> >> > How about shared data instances ? How does that work ?
> >> >
> >> > ...as a side note...creating our own module system
> now - I
> feel that is a very bad direction :(
> >> > Might as well adopt osgi plugin system if you want
> this
> kind of separation ?
> >> >
> >> > /max
> >> >
> >> > >
> >> > > ~Lincoln
> >> > >
> >> > > On Mon, Apr 18, 2011 at 11:56 AM, Max Rydahl
> Andersen
> <max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>
> <mailto:max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>>> wrote:
> >> > >
> >> > > On Apr 18, 2011, at 15:15, Lincoln Baxter, III
> wrote:
> >> > >
> >> > > > "if there is a standard location for dependencies"
> >> > > >
> >> > > > What do you mean?
> >> > >
> >> > > Your "standard" for shading is that you put all
> classes
> into the plugin.jar.
> >> > >
> >> > > A "standard" for dependencies for a plugin.jar
> could be
> "next to the plugin.jar".
> >> > >
> >> > > Would still have the problem of overlapping jars but
> then at least its easier to see where the
> duplication is.
> >> > >
> >> > > /max
> >> > >
> >> > > >
> >> > > > Thx,
> >> > > > ~Lincoln
> >> > > >
> >> > > > On Mon, Apr 18, 2011 at 6:20 AM, Max Rydahl
> Andersen
> <max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>
> <mailto:max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>>> wrote:
> >> > > >
> >> > > > > I was thinking we might already be able to
> do that
> using the existing pom.xml metadata that's
> stored in the
> artifact itself, or is that too tricky?
> >> > > >
> >> > > > if there is a standard location for
> dependencies then
> it should be fine - at least better than shading ;)
> >> > > >
> >> > > > /max
> >> > > >
> >> > > > >
> >> > > > > On Mon, Apr 11, 2011 at 6:51 PM, Max Andersen
> <manderse at redhat.com <mailto:manderse at redhat.com>
> <mailto:manderse at redhat.com
> <mailto:manderse at redhat.com>>> wrote:
> >> > > > > I was thinking Plugin jar having references to
> dependent jars via manifest.mf
> >> > > > >
> >> > > > > /max (sent from my phone)
> >> > > > >
> >> > > > >
> >> > > > > On 12/04/2011, at 00.39, "Lincoln Baxter, III"
> <lincolnbaxter at gmail.com
> <mailto:lincolnbaxter at gmail.com>
> <mailto:lincolnbaxter at gmail.com
> <mailto:lincolnbaxter at gmail.com>>> wrote:
> >> > > > >
> >> > > > >> Can you give an example of how you would
> bundle the
> JARs? (Just put them in /META-INF/dependencies/
> ... ?) And would
> that not cause just as many class conflicts? If you
> shade/relocate then the deps *should be*
> completely isolated.
> >> > > > >>
> >> > > > >> On Mon, Apr 11, 2011 at 4:11 PM, Max Rydahl
> Andersen <max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>
> <mailto:max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>>> wrote:
> >> > > > >> well, recommending just bundling jars would
> be a
> better approach than shading IMO.
> >> > > > >>
> >> > > > >> /max
> >> > > > >>
> >> > > > >> On Apr 11, 2011, at 16:00, Lincoln Baxter,
> III wrote:
> >> > > > >>
> >> > > > >> > Yeah, shading is currently the recommended
> approach. Conflicts should be avoided by using
> relocations. I
> know this is... not a great method, but for now
> it's all we've
> got. Open to suggestions.
> >> > > > >> >
> >> > > > >> > ~Lincoln
> >> > > > >> >
> >> > > > >> > On Mon, Apr 11, 2011 at 3:41 AM, Max Rydahl
> Andersen <max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>
> <mailto:max.andersen at redhat.com
> <mailto:max.andersen at redhat.com>>> wrote:
> >> > > > >> > Heya,
> >> > > > >> >
> >> > > > >> > Lincoln, I just saw your commits to
> hibernattools
> plugin at
>
> (https://github.com/forge/plugin-hibernate-tools/commit/8b208b4a8e79dbb8a01d10d266ee81afd2cf7106)
> >> > > > >> >
> >> > > > >> > Is shading of jars really the recommended
> approach for plugins in Forge ?
> >> > > > >> >
> >> > > > >> > How are you going to share/avoid
> collisions of
> libraries across plugins if they need to bundle
> via shading ?
> >> > > > >> >
> >> > > > >> > /max
> >> > > > >> > http://about.me/maxandersen
> >> > > > >> >
> >> > > > >> >
> >> > > > >> >
> >> > > > >> >
> >> > > > >> >
> _______________________________________________
> >> > > > >> > forge-dev mailing list
> >> > > > >> > forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>
> <mailto:forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>>
>
> >> > > > >> >
> https://lists.jboss.org/mailman/listinfo/forge-dev
> >> > > > >> >
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > --
> >> > > > >> > Lincoln Baxter, III
> >> > > > >> > http://ocpsoft.com
> >> > > > >> > http://scrumshark.com
> >> > > > >> > "Keep it Simple"
> >> > > > >>
> >> > > > >> /max
> >> > > > >> http://about.me/maxandersen
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> --
> >> > > > >> Lincoln Baxter, III
> >> > > > >> http://ocpsoft.com
> >> > > > >> http://scrumshark.com
> >> > > > >> "Keep it Simple"
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Lincoln Baxter, III
> >> > > > > http://ocpsoft.com
> >> > > > > http://scrumshark.com
> >> > > > > "Keep it Simple"
> >> > > >
> >> > > > /max
> >> > > > http://about.me/maxandersen
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Lincoln Baxter, III
> >> > > > http://ocpsoft.com
> >> > > > http://scrumshark.com
> >> > > > "Keep it Simple"
> >> > >
> >> > > /max
> >> > > http://about.me/maxandersen
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Lincoln Baxter, III
> >> > > http://ocpsoft.com
> >> > > http://scrumshark.com
> >> > > "Keep it Simple"
> >> >
> >> > /max
> >> > http://about.me/maxandersen
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Lincoln Baxter, III
> >> > http://ocpsoft.com
> >> > http://scrumshark.com
> >> > "Keep it Simple"
> >>
> >> /max
> >> http://about.me/maxandersen
> >>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Lincoln Baxter, III
> >> http://ocpsoft.com
> >> http://scrumshark.com
> >> "Keep it Simple"
> >
> >
> >
> > --
> > Lincoln Baxter, III
> > http://ocpsoft.com
> > http://scrumshark.com
> > "Keep it Simple"
> >
> > _______________________________________________
> > forge-dev mailing list
> > forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>
> <mailto:forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>>
>
> > https://lists.jboss.org/mailman/listinfo/forge-dev
> >
> >
> > _______________________________________________
> > forge-dev mailing list
> > forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>
> <mailto:forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>>
>
> > https://lists.jboss.org/mailman/listinfo/forge-dev
>
> /max
> http://about.me/maxandersen
>
>
>
>
> _______________________________________________
> forge-dev mailing list
> forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>
> <mailto:forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>>
>
> https://lists.jboss.org/mailman/listinfo/forge-dev
>
>
>
> _______________________________________________
> forge-dev mailing list
> forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>
> <mailto:forge-dev at lists.jboss.org
> <mailto:forge-dev at lists.jboss.org>>
>
> https://lists.jboss.org/mailman/listinfo/forge-dev
>
>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.com
> http://scrumshark.com
> "Keep it Simple"
>
>
>
> --
> - DML
>
>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.com
> http://scrumshark.com
> "Keep it Simple"
>
>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.com
> http://scrumshark.com
> "Keep it Simple"
>
>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.com
> http://scrumshark.com
> "Keep it Simple"
--
- DML
More information about the forge-dev
mailing list