[forge-dev] shading!?

Wed Apr 20 12:05:42 EDT 2011

Adding mike, who is apparently not on forge-dev... fix that :)

On Wed, Apr 20, 2011 at 11:56 AM, Lincoln Baxter, III <
lincolnbaxter at gmail.com> wrote:

> I agree with both of you guys, which is why I am soliciting ideas for this
> now.
>
> I can also say that we do not have one big Classpath right now (unless I am
> horribly mistaken.) Forge already supports classpath isolation of individual
> plugins, so we're already half-way there. If my understanding of
> classloaders is at all right, It would not be too much work to implement a
> maven resolver and additional level of Composite classloaders to achieve
> this goal.
>
> I don't believe we need to pull in a heavyweight module system like OSGI,
> or force developers to do even the simple configuration for something like
> JBoss modules (which afaik, does not have a way of packaging the actual
> dependencies with the module, or calling out to Maven for dependencies -
> which forge can already do.)
>
> *This is what we have right now:* (see today.png)
>
> Notice how the Plugin ClassLoaders may reference Forge APIs, but the
> plugins may not access each other's classloaders because there is no link
> from Forge APIs to the Plugins.
>
> *And this is all I think it should take to do what I am suggesting:* (see
> tomorrow.png)
>
> If it is technically possible to achieve with JBoss modules, via adding in
> parsing the POM and including dependencies automatically, then I am all for
> it, but from what I understand, this would not really be a 1-1 mapping and
> will not be trivial. I know my understanding of ClassLoaders may be a bit
> Naive, but I don't think that we need to use JBoss modules (or OSGi) to
> achieve this goal of automatic downloading and loading of plugin dependency
> JARs.
>
> We certainly do not need the ability (yet) to drop an individual module
> while Forge is running, because Weld does not support this kind of
> disruption in the BeanStore. If you remove a JAR while Weld is running, you
> get a big fiery explosion. Same for loading a single JAR, you have to
> restart Weld because it will not be aware of the changes, and there is not a
> good API to make it so. It still has to build the bean graph all over again.
> That is one of the big reasons to use a fully-fledged module system, and we
> don't have that requirement. Forge simply dumps all classloaders (until JDK
> 7 we can't close them easily, yeah memory leaks, whatever, not really
> important right now,) then builds a new network of ClassLoaders and starts
> over.
>
> However, if we can extend JBoss modules with Maven support in this way, and
> it's something that David would be interested in, then I think that's one
> big reason to do it that way. Until then, if we can do it quickly my way,
> I'd like to get it working sooner rather than later. Please please tell me
> (I'm sure I don't even have to ask) if there are gaping technical holes in
> what I am proposing, but if there are not, then I'm going to stick to this
> position until someone bursts my bubble.
>
> ~Lincoln
>
>
>
> On Wed, Apr 20, 2011 at 9:33 AM, Paul Bakker <paul.bakker.nl at gmail.com>wrote:
>
>> Yeah, I'm afraid that's right. I think the module system integration
>> should get very high priority before we reach a 1.0 version. It's something
>> that affects the whole way the core is built up, and it will be very
>> difficult to change that later on.
>>
>> Paul
>>
>>
>>
>>
>> On Wed, Apr 20, 2011 at 3:17 PM, Max Rydahl Andersen <
>> max.andersen at redhat.com> wrote:
>>
>>> +1
>>>
>>> I agree with what you want to do Lincoln, but don't see how that can be
>>> made to happen without a module system.
>>>
>>> Until then everything just will have to live in one big classpath.
>>>
>>> /max
>>>
>>> On Apr 20, 2011, at 14:24, Paul Bakker wrote:
>>>
>>> > And that's why a module system is needed. If people can just add
>>> dependencies there will be duplicate (possibly multiple versions)
>>> dependencies. Each plugin should be in a separate classloader, and each
>>> dependency should be too so that a plugin never breaks other plugins or
>>> Forge itself. It would be a big limitation if plugins can't use libraries
>>> otherwise.
>>> >
>>> > Paul
>>> >
>>> > On Tue, Apr 19, 2011 at 6:14 PM, Lincoln Baxter, III <
>>> lincolnbaxter at gmail.com> wrote:
>>> > That's right, but what I'm saying is that I don't want developers to be
>>> responsible for anything but *their code* -- if they have dependencies,
>>> those depenencies will be fetched for them (or somehow bundled in the JAR
>>> file itself, which is certainly possible, however not my preference.)
>>> >
>>> > If you could write a plugin, reference dependencies in your POM, and
>>> have everything *Just Work* don't you think that would be a friendlier
>>> experience?
>>> >
>>> > ~Lincoln
>>> >
>>> >
>>> > On Mon, Apr 18, 2011 at 5:55 PM, Max Andersen <manderse at redhat.com>
>>> wrote:
>>> > But if your plugin uses multiple jars it is not one jar.
>>> >
>>> >
>>> > /max (sent from my phone)
>>> >
>>> >
>>> > On 18/04/2011, at 19.06, "Lincoln Baxter, III" <
>>> lincolnbaxter at gmail.com> wrote:
>>> >
>>> >> That doesn't solve the problem of having to drop jar files onto the
>>> classpath in order for plugins to work. I want one JAR per plugin.
>>> >>
>>> >> ~Lincoln
>>> >>
>>> >> On Mon, Apr 18, 2011 at 12:59 PM, Max Rydahl Andersen <
>>> max.andersen at redhat.com> wrote:
>>> >>
>>> >> On Apr 18, 2011, at 18:57, Lincoln Baxter, III wrote:
>>> >>
>>> >> > I don't want to force plugin-developers to create modules for every
>>> dependency that their plugin requires. That's why I've been avoiding OSGI or
>>> JBoss Modules.
>>> >>
>>> >> But then you shouldn't be forcing them to shade either - you should
>>> just have one global classloader for the plugins then.
>>> >>
>>> >> /max
>>> >>
>>> >> >
>>> >> > ~Lincoln
>>> >> >
>>> >> > On Mon, Apr 18, 2011 at 12:37 PM, Max Rydahl Andersen <
>>> max.andersen at redhat.com> wrote:
>>> >> >
>>> >> > > What do you think about using Maven APIs to inspect the POM and
>>> fetch dependencies dynamically for each plugin, then isolate them in the
>>> plugin's classloader?
>>> >> >
>>> >> > Why not just load them in to one classloader so you don't have
>>> collisions when there are mixed dependencies on Forge it self ?
>>> >> >
>>> >> > How about shared data instances ? How does that work ?
>>> >> >
>>> >> > ...as a side note...creating our own module system now - I feel that
>>> is a very bad direction :(
>>> >> > Might as well adopt osgi plugin system if you want this kind of
>>> separation ?
>>> >> >
>>> >> > /max
>>> >> >
>>> >> > >
>>> >> > > ~Lincoln
>>> >> > >
>>> >> > > On Mon, Apr 18, 2011 at 11:56 AM, Max Rydahl Andersen <
>>> max.andersen at redhat.com> wrote:
>>> >> > >
>>> >> > > On Apr 18, 2011, at 15:15, Lincoln Baxter, III wrote:
>>> >> > >
>>> >> > > > "if there is a standard location for dependencies"
>>> >> > > >
>>> >> > > > What do you mean?
>>> >> > >
>>> >> > > Your "standard" for shading is that you put all classes into the
>>> plugin.jar.
>>> >> > >
>>> >> > > A "standard" for dependencies for a plugin.jar could be "next to
>>> the plugin.jar".
>>> >> > >
>>> >> > > Would still have the problem of overlapping jars but then at least
>>> its easier to see where the duplication is.
>>> >> > >
>>> >> > > /max
>>> >> > >
>>> >> > > >
>>> >> > > > Thx,
>>> >> > > > ~Lincoln
>>> >> > > >
>>> >> > > > On Mon, Apr 18, 2011 at 6:20 AM, Max Rydahl Andersen <
>>> max.andersen at redhat.com> wrote:
>>> >> > > >
>>> >> > > > > I was thinking we might already be able to do that using the
>>> existing pom.xml metadata that's stored in the artifact itself, or is that
>>> too tricky?
>>> >> > > >
>>> >> > > > if there is a standard location for dependencies then it should
>>> be fine - at least better than shading ;)
>>> >> > > >
>>> >> > > > /max
>>> >> > > >
>>> >> > > > >
>>> >> > > > > On Mon, Apr 11, 2011 at 6:51 PM, Max Andersen <
>>> manderse at redhat.com> wrote:
>>> >> > > > > I was thinking Plugin jar having references to dependent jars
>>> via manifest.mf
>>> >> > > > >
>>> >> > > > > /max (sent from my phone)
>>> >> > > > >
>>> >> > > > >
>>> >> > > > > On 12/04/2011, at 00.39, "Lincoln Baxter, III" <
>>> lincolnbaxter at gmail.com> wrote:
>>> >> > > > >
>>> >> > > > >> Can you give an example of how you would bundle the JARs?
>>> (Just put them in /META-INF/dependencies/ ... ?) And would that not cause
>>> just as many class conflicts? If you shade/relocate then the deps *should
>>> be* completely isolated.
>>> >> > > > >>
>>> >> > > > >> On Mon, Apr 11, 2011 at 4:11 PM, Max Rydahl Andersen <
>>> max.andersen at redhat.com> wrote:
>>> >> > > > >> well, recommending just bundling jars would be a better
>>> approach than shading IMO.
>>> >> > > > >>
>>> >> > > > >> /max
>>> >> > > > >>
>>> >> > > > >> On Apr 11, 2011, at 16:00, Lincoln Baxter, III wrote:
>>> >> > > > >>
>>> >> > > > >> > Yeah, shading is currently the recommended approach.
>>> Conflicts should be avoided by using relocations. I know this is... not a
>>> great method, but for now it's all we've got. Open to suggestions.
>>> >> > > > >> >
>>> >> > > > >> > ~Lincoln
>>> >> > > > >> >
>>> >> > > > >> > On Mon, Apr 11, 2011 at 3:41 AM, Max Rydahl Andersen <
>>> max.andersen at redhat.com> wrote:
>>> >> > > > >> > Heya,
>>> >> > > > >> >
>>> >> > > > >> > Lincoln, I just saw your commits to hibernattools plugin at
>>> (
>>> https://github.com/forge/plugin-hibernate-tools/commit/8b208b4a8e79dbb8a01d10d266ee81afd2cf7106
>>> )
>>> >> > > > >> >
>>> >> > > > >> > Is shading of jars really the recommended approach for
>>> plugins in Forge ?
>>> >> > > > >> >
>>> >> > > > >> > How are you going to share/avoid collisions of libraries
>>> across plugins if they need to bundle via shading ?
>>> >> > > > >> >
>>> >> > > > >> > /max
>>> >> > > > >> > http://about.me/maxandersen
>>> >> > > > >> >
>>> >> > > > >> >
>>> >> > > > >> >
>>> >> > > > >> >
>>> >> > > > >> > _______________________________________________
>>> >> > > > >> > forge-dev mailing list
>>> >> > > > >> > forge-dev at lists.jboss.org
>>> >> > > > >> > https://lists.jboss.org/mailman/listinfo/forge-dev
>>> >> > > > >> >
>>> >> > > > >> >
>>> >> > > > >> >
>>> >> > > > >> > --
>>> >> > > > >> > Lincoln Baxter, III
>>> >> > > > >> > http://ocpsoft.com
>>> >> > > > >> > http://scrumshark.com
>>> >> > > > >> > "Keep it Simple"
>>> >> > > > >>
>>> >> > > > >> /max
>>> >> > > > >> http://about.me/maxandersen
>>> >> > > > >>
>>> >> > > > >>
>>> >> > > > >>
>>> >> > > > >>
>>> >> > > > >>
>>> >> > > > >>
>>> >> > > > >> --
>>> >> > > > >> Lincoln Baxter, III
>>> >> > > > >> http://ocpsoft.com
>>> >> > > > >> http://scrumshark.com
>>> >> > > > >> "Keep it Simple"
>>> >> > > > >
>>> >> > > > >
>>> >> > > > >
>>> >> > > > > --
>>> >> > > > > Lincoln Baxter, III
>>> >> > > > > http://ocpsoft.com
>>> >> > > > > http://scrumshark.com
>>> >> > > > > "Keep it Simple"
>>> >> > > >
>>> >> > > > /max
>>> >> > > > http://about.me/maxandersen
>>> >> > > >
>>> >> > > >
>>> >> > > >
>>> >> > > >
>>> >> > > >
>>> >> > > >
>>> >> > > > --
>>> >> > > > Lincoln Baxter, III
>>> >> > > > http://ocpsoft.com
>>> >> > > > http://scrumshark.com
>>> >> > > > "Keep it Simple"
>>> >> > >
>>> >> > > /max
>>> >> > > http://about.me/maxandersen
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > > --
>>> >> > > Lincoln Baxter, III
>>> >> > > http://ocpsoft.com
>>> >> > > http://scrumshark.com
>>> >> > > "Keep it Simple"
>>> >> >
>>> >> > /max
>>> >> > http://about.me/maxandersen
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Lincoln Baxter, III
>>> >> > http://ocpsoft.com
>>> >> > http://scrumshark.com
>>> >> > "Keep it Simple"
>>> >>
>>> >> /max
>>> >> http://about.me/maxandersen
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Lincoln Baxter, III
>>> >> http://ocpsoft.com
>>> >> http://scrumshark.com
>>> >> "Keep it Simple"
>>> >
>>> >
>>> >
>>> > --
>>> > Lincoln Baxter, III
>>> > http://ocpsoft.com
>>> > http://scrumshark.com
>>> > "Keep it Simple"
>>> >
>>> > _______________________________________________
>>> > forge-dev mailing list
>>> > forge-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/forge-dev
>>> >
>>> >
>>> > _______________________________________________
>>> > forge-dev mailing list
>>> > forge-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/forge-dev
>>>
>>> /max
>>> http://about.me/maxandersen
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> forge-dev mailing list
>>> forge-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>>
>>
>>
>> _______________________________________________
>> forge-dev mailing list
>> forge-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>
>>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.com
> http://scrumshark.com
> "Keep it Simple"
>

-- 
Lincoln Baxter, III
http://ocpsoft.com
http://scrumshark.com
"Keep it Simple"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/forge-dev/attachments/20110420/d03cd784/attachment-0001.html 