[jboss-dev] Refactoring packaging/packages to reduce "signed jar groups"

[Carlo de Wolf]

On 02/25/2010 01:46 AM, Jesper Pedersen wrote:
> Hey Mike,
>    
Hi all,

And down we go... :-)
> On Wednesday 24 February 2010 17:50:22 Mike Clark wrote:
>    
>> The general purpose of this note is to explore refactoring some of our
>> jars to reduce some problems that have come about due to our signing of
>> jars.
>>
>> So, let me start with a description of the problem.   Essentially, the
>> core issue is around the simple replacement of an EAP signed jar with an
>> unsigned jar.  While we have a system in place to have QE get one-off
>> patches signed in a relatively responsive manner, we are still left with
>> cases such as a diagnostic jar created by support to track down
>> problems, or a customer that, understanding that it would not be
>> supported, wishes to make a change for their system (one of the
>> advantages of OSS, of course).
>>
>> The problem with such a substitution is the fact that a classloader will
>> not load classes from a packages that appear in differently signed
>> jars.  So, for example, even though class org.jboss.A may only occur in
>> an unsigned A.jar, loading it will cause a security exception if class
>> org.jboss.B was loaded from a signed B.jar.  Now, the first step to
>> resolve this would be to replace both A.jar and B.jar with unsigned
>> versions.  But, we then often run into a transitive entanglement.  B.jar
>> may contain classes from package org.jboss, but also org.hibernate.  So,
>> you have to have unsigned versions of C.jar and D.jar that contain the
>> hibernate packages.  But, they have an org.apache package, and so on.
>>
>> Using TattleTale with some custom reports, I generated "signed jar
>> groups" identifying clusters of jars that would need to all be replaced
>> as unsigned if any one is unsigned.  For the largest group, I created a
>> graph of the dependencies shown in
>> http://community.jboss.org/wiki/SignedJarDependenciesExample
>> (getopt_jar_Group.png, named by the first jar listed in the group from
>> the report.)
>>
>> The figure shows the jars with a line connecting them if they have one
>> or more packages in common (I also generate detailed plots with a
>> labelled line for each package in common if anyone wants to see them,
>> but for the "getopt" group, the detailed plot is quite large and hard to
>> read.)  Clearly, lots of unrelated jars are wound up in this group.
>> But, we can chip away at this with a bit of analysis.  I'll give three
>> examples.
>>
>> First, consider the set of jar files just to the right of the center
>> grouped around jboss-ejb3-core.jar.  This set of jars is tied into the
>> rest of the graph through the connection between
>> jboss-as-ejb3-deployer.jar and jboss.jar. The package that ties them is
>> org.jboss.as.javaee.  In this case there appears to be only two classes
>> involved.  In jboss.jar there is
>> org.jboss.as.javaee.SimpleJavaEEModuleIdentifier and in
>> jboss-as-ejb3-deployer.jar there is
>> org.jboss.as.javaee.SimpleJavaEEModuleInformer (which has an inner
>> class).  So, in this case, we can break off a chunk of the signed jar
>> group by, for example, changing a package name:
>> org.jboss.as.javaee.ejb3.SimpleJavaEEModuleInformer
>>
>> Next, from the graph, we can see that there is a group of jar files on
>> the far left that is tied in by a connection between the
>> jboss-system-jmx.jar and the jboss.jar.  In this case, the package is
>> org.jboss.deployment.  There are several classes in jboss.jar and a
>> handful in jboss-system-jmx.jar that are in this package.  So, options
>> might include renaming the package for the classes in the
>> jboss-system-jmx.jar, moving them to jboss.jar, or separating the
>> org.jboss.deployment classes between them into a new jar:
>> jboss-deployment.jar.
>>
>> Finally, staying on the far left of the graph we see that run.jar and
>> shutdown.jar are tied into some third party jars such as getopt.jar,
>> hsqldb.jar and jdom-1.0.jar.  These are due to use of classes in the
>> default package.  Further, the default package is included because the
>> contents of the getopt.jar are incorporated into run.jar and
>> shutdown.jar.  So, this connection to third party jars can be broken out
>> by removing the GetOpt related classes from run.jar and shutdown.jar and
>> relying on the Class-Path: attribute in their manifests.
>>
>> So, in the end, it would seem to be beneficial to refactor package
>> locations such that there is not such widespread entanglement.  From the
>> signed jar perspective, it would be best if each package only appeared
>> in one jar.  Of course, there are often engineering reasons to separate
>> classes in the same package into different jars.  However, I think it's
>> not unreasonable that these should be relatively small in number and
>> they should be functionally coherent.  At any rate, it would appear to
>> be something we could attack incrementally as indicated by the three
>> examples above.
>>
>> Originally, when discussing this with other folks, we thought perhaps we
>> should just open jiras to address the problem.  But, on considering
>> that, it seemed that perhaps a discussion on a larger game plan would be
>> a better approach, even if it just ended up with the original plan of
>> opening some jiras to "chip away" at the problem.
>>
>> So, I apologize for a long, dense note.  For those of you that made it
>> through, any thoughts, suggestions?
>>
>>      
> It really boils down to
>
>   https://jira.jboss.org/jira/browse/JBAS-1796
>
> See Adrian's comments on the issue.
>    
This has little to do with JBAS-1796. It's simply a package namespace 
conflict which is not allowed in a signed jars environment. This was 
expected.

In essence both classes are in the wrong package, because the different 
modules should only govern their own namespace. So the proper solution 
would be to move org.jboss.as.javaee.SimpleJavaEEModuleIdentifier to 
org.jboss.as.server.javaee and 
org.jboss.as.javaee.SimpleJavaEEModuleInformer to org.jboss.as.ejb3.javaee.
For the moment lets only do the later.

What is part of JBAS-1796 is the fact that some modules are unscoped and 
bleed left and right.
We really need to hack apart the modules, so that we're forced to create 
the proper integration points.
In essence we have a base runtime (Reloaded) that allows modules to run 
on their own with possible mocks in place.

Carlo

> The latest Tattletale release have support for both Ant and Maven style build
> environments, so its up to the project leads to integrate the tool into their
> environment and start looking at the dependencies. The latest release also
> features the ability to filter 'incorrect' results, so getting clean reports
> should be doable.
>
> I have added
>
>   https://jira.jboss.org/jira/browse/TTALE-89
>
> which could help to identify how JARs better could be split into a better
> structure - still open though. Feel free to hack away :)
>
> For now, I would say if you have certain requirements just open JIRAs for the
> projects in question - or better: create a forum thread in the projects
> developer forum.
>
> Best regards,
>   Jesper
> _______________________________________________
> jboss-development mailing list
> jboss-development at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-development
>