Re: [jboss-dev-forums] [JBoss AS7 Development] - Modular Serialization
by Jason Greene
Jason Greene [http://community.jboss.org/people/jason.greene] commented on the document
"Modular Serialization"
To view all comments on this document, visit: http://community.jboss.org/docs/DOC-17244#comment-7738
--------------------------------------------------
I think you are referring to the scenario in https://issues.jboss.org/browse/HORNETQ-782 HORNETQ-782. Is that correct?
This case is a form of Problem A. The difference is that java.lang.Object is the shared super class, and the sun jdk is "shared.jar". I had commented on the issue that it was "near" invalid, since there isn't much use you can do with java.lang.Object. You could use reflection, but typing would be superior. Problem A is the "legitimate" version where you have a real sensible contract that works fine in the local case, not so great with serialization.
Both solutions will work in even in the strang java.lang.Object case. The difference mentioned above is that it's a choice between putting the burden on the developer or the implementer. Since we have JBoss Marshalling, which does all of the hard work, I think that burden is pretty small. I think the challenge in the HORNETQ case is just the protocol changes and handling older clients.
--------------------------------------------------
14 years, 3 months
Re: [jboss-dev-forums] [JBoss AS7 Development] - Modular Serialization
by Yong Hao Gao
Yong Hao Gao [http://community.jboss.org/people/gaohoward] commented on the document
"Modular Serialization"
To view all comments on this document, visit: http://community.jboss.org/docs/DOC-17244#comment-7733
--------------------------------------------------
Refering to Problem C, considering the following case:
1. The framework load Foo class, and the application new a Foo object.
2. The application uses services provided by the thirdparty.jar, which is loaded by the framework too.
3. The application then pass Foo object to thirdparty service where it is serialized. There should be no problem because the serialization doesn't need classloader to participate.
4. Later application reqests the Foo object from the 3rd party service. So the 3rd party tries to deserialize the Foo object, but unfortunately the classloader used to load Foo class is not seen by 3rdparty service because the application explicitly asked the framework to do so in its deployment config.
It's all happened within one VM. If the framework doesn't deploy the app as an isolated component, the 3rdparty will have no problem deserializing it.
In that case the 3rd party doesn't know beforehand how the serialized object's class is loaded and so is completely innocent of the problem. But the application and the framework are fully aware of the problem, and should do something properly to avoid this.
--------------------------------------------------
14 years, 3 months
[JBoss AS7 Development] - Module Compatible Classloading Guide
by Jason Greene
Jason Greene [http://community.jboss.org/people/jason.greene] modified the document:
"Module Compatible Classloading Guide"
To view the document, visit: http://community.jboss.org/docs/DOC-16701
--------------------------------------------------------------
h3. Introduction
Modular classloading has a much greater focus on isolation and separation than traditional heirarchical models, so it is far less forgiving about using the "wrong" classloader. Unfortunately, this means that any dynamic classloading code you have written in the past may need to be updated to play nicely in a modular environment. However, the good news is that code that is module compatible is more correct, and therefore less error prone in traditional environments. This article provides some background infromation along with some pointers on how to make this transition.
h3. Common Terms
This article assumes that the following terms are well understood.
*Defining Classloade*r - The classloader which loaded the calling class. This is what is used when you use Class.forName() without specifying a classloader. It's also what loads classes that you reference statically (e.g. via an import). It can be obtained by calling *getClass().getClassLoader()* on the respective class.
*Thread Context Classloader (TCCL)* - The classloader which has been assigned to the calling thread. It's meaning is specific to the environment it was assigned in. In Java EE code, the TCCL is always the classloader of the relevant deployment. *+The TCCL is the most commonly misused classloader.+*
*Dynamic Classloading* - Loading a class by name using a ClassLoader. This is typically used for pluggable architectures that discover some kind of extension. An example is an API framework that supports multiple implementations/providers (e.g. JAXP). It dynamically loads the provider by name using some kind of service discovery mechanism (a file in META-INF/services, a system property, etc). Another example is application server code which needs to operate against a deployment's classes. Since they are logically isolated, and may even be hot-deployed, static linking (imports) can not be used.
*JDK Service Provider Loading* - The common pattern used by Java SE and Java EE frameworks to load different providers / implementations of an API. This is done by checking a system property, a special properties file, and more commonly looking for files in META-INF/services. See the http://download.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html ServiceLoader javadoc for more info on how the process works.
*Module* - A single consistent class loading namespace. It encompases classes/resources directly available in the module (contents of one or more jar(s)) local to the module, as well as all module dependencies defined on the module. Every module has it's own ClassLoader which it uses to load all classes that can be loaded from the module.
*Deployment* - A deployment is a user provided package of components. In AS7, every deployment has a Module. In some cases a deployment may contain nested deployments or other artifacts. Those are often represented by a module of their own.
*System Classloader* - Also commonly referred to as the "application classloader". It represents everything that is defined on java.class.path, and commonly includes java extensions and some jdk classes.
h3. Picking the "Right" Classloader
The key to loading classes in a modular environment is to use the correct classloader. In the past many frameworks, including many Java standard APIs, expect that the users classes (or the deployment classes) and the framework classes are accessible via the same classloader.* +The problem with this assumption is that it prevents proper isolation of the framework implementation classes and a user's classes.+* In many cases though it would work in a heirarchical model (as long as there was no name conflicts), since class sharing is done by inheritance rather than explicit links as is the case in a modular environment. This single bad assumption makes it impossible to achieve capabilities like allowing a user to use their own XML parser.
The correct way to pick a ClassLoader is to always use the one that is directly associated with the class you are trying to load. So if a framework wants to load a class from one of it's dependencies +*then it should use it's defining classloader.*+ Likewise, If the framework wants to load a class from a deployment, then it should use the deployment's classloader. In some cases, like in the case of service provider discovery, a framework might want to look for a particular resource in both it's classloader (say a default provider), and the user Classloader. Under this scenario the framework should check both ClassLoaders following the order that best suits it's needs.
h3. *FRAMEWORKS, NEVER LOAD YOUR CLASSES FROM THE TCCL!!*
If you expect portability between different classloading environments, it is always wrong for a framework to load it's classes from the TCCL. +*Instead use the right classloader, which is most often the defining classloader.*+ If the defining classloader is not the right classloader, then look for a way to get the right loader and use it to load classes. Even if you have an inheritance model with a heirarchy that sees the classes you want, it is always more correct to pick the classloader that "owns" the classes.
The TCCL should be used with extreme care. EE code is required to see the defining deployments classloader as the TCCL. Beware though that other frameworks may have different rules, and may set it to another value when executing in their context. I+*f you have access to the classloader you want it's always better to use it over the TCCL.*+
Note that AS7 subsystems that are executing within any kind of EE context can safely asssume that the TCCL will point to the deployment classloader. They should, however, never attempt to load AS7 code from the deployment classloader.
h3. What To Export/Expose
The only thing that a module should ever expose is classes that make up it's API contracts. In most cases this means that an AS7 module should not export it's dependencies. This allows for the module and it's calling module to use completely different versions of the same dependency. Deployments can, for example, use commons-bah 1.2 and AS7 code can happily use commons-blah 1.0.
h3. How to deal with "Bad" dependencies
If your module/deployment/framework depends on something that isn't compatible with a modular environment. Then you have a couple of options that may workaround the issue. However in some cases the only workable solution is to patch the bad framework.
Common workarounds include:
* *Swapping TCCL* - If a call to a framework method incorrectly uses the TCCL to load it's classes/resources, and the same call does not need to see deployment classes, then temporarily set the TCCL to the framework's defining classloader. (e.g. Thread.currentThread().setContextClassLoader(frameworkClass.getClass().getClassLoader()
* *Prefer calls which have a CL argument* - If you can pass a classloader, prefer that to calls which try to guess (usually picking the TCCL) if possible.
* *Heirarchical Emulation -* This is a worst case workaround that should only be used as a last resort. In the unfortunate case where a framework expects both deployment classes and it's own classes to be on the same classloader, a custom classloader can be created which delegates first to the deployment classsloader and second to frameworks defining classloader. This classloader can the be swapped to the TCCL as described above, or somehow passed to the framework via a mechnism it exposes. This has the undesirable effect of a possible conflict between a user's implementation choices and the frameworks. However, it at least does not pollute the deployment module.
If you have a bad dependency then file a bug to get it fixed/replaced. In particularly if you have to use heirarchical emulation.
h3. How to use JAXP and other service provider APIs from AS/Framework code
EE applications can safely use the no-arg newInstance() methods provided by JAXP. However framework code must either not use them, or use the swapping TCCL workaround described in the "bad" dependencies section above. This ensures that either the container's xml parser is used or the parser explictly imported by the frameworks module (requires a service import). Not doing this will result in the deployments xml parser being used (since the deployment CL is often the TCCL), which may be incompatible.
h3. Serialization
See the aritcle on http://community.jboss.org/docs/DOC-17244 Modular Serialization.
--------------------------------------------------------------
Comment by going to Community
[http://community.jboss.org/docs/DOC-16701]
Create a new document in JBoss AS7 Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=102&co...]
14 years, 3 months
[JBoss Transactions Development] - XTSTestingCurrentStatus
by Ivo Studensky
Ivo Studensky [http://community.jboss.org/people/istudens] modified the document:
"XTSTestingCurrentStatus"
To view the document, visit: http://community.jboss.org/docs/DOC-17242
--------------------------------------------------------------
h1. XTS Testing
h1. Current Status and Roadmap
h1.
This Document describes the current status of the XTS tests and what technology they use. There are currently three sets of tests, Unit, Interop and recovery. Each of these is described in turn, and a list of required improvements is presented.
All improvements are targeted at Narayana 5.x unless specified otherwise.
h2. Unit tests
Each XTS component has a set of unit tests. These tests need to be ran within an instance of JBoss AS. These tests are fully automated by an Ant script ran by Hudson.
h3. Improvements
*High Priority (EAP 6.0)*
1. *Ensure AS 7 compatibility.* See issue https://issues.jboss.org/browse/JBTM-900 JBTM-900.
2. *Move into the AS test suite*. https://issues.jboss.org/browse/JBQA-5191 https://issues.jboss.org/browse/JBQA-5191. This will ensure that changes to components, we depend upon (like JbossWS), that break XTS are spotted at QE time rather than after release. (EAP 6 requirement)
3. *Additional Testing*. XTS Demo tests ( https://issues.jboss.org/browse/JBQA-5194 https://issues.jboss.org/browse/JBQA-5194)
*Medium Priority (Post EAP 6)*
1. *Automate Emma and collate results*. Emma should be used to obtain coverage stats on the test run. Emma, produces individual reports per test, by default. It would be better to have these reports combined as we are concerned with the overall test coverage, rather than the coverage of each test. Ideally we would combine coverage stats over all sets of tests (unit, interop and recovery). Next step would be to improve the coverage where necessary.
*Low Priority* *(Post EAP 6)*
1. *Migrate to Maven*. For consistency with rest of the Narayna project
2. *Update to use Arquillian*. This would mean that that they can be ran from anywhere that can run JUnit tests, such as an IDE or maven. It would also automate the app server lifecycle and test deployment.
3. *Remove home-brew SOAP stack*. In the past these tests used a mock/simple SOAP stack developed specifically for the tests. This stack is no longer used as the tests run within JBoss AS. This code is redundant and should be removed.
h2. Interop Tests
We have two sets of interop tests that live in "XTS/interop". These are built with ant. They are each ran by deploying them as a war to a single JBoss instance which deploys the services needed by the test. This war also exposes a web interface that on request runs the tests and relays the results in the http response.
This process is automated by using ant to deploy the war and then making the http request and validating the response. See here in code for scripts: “XTS/localjunit/run-interop-tests.xml” and “XTS/localjunit/run-tests.xml”
h3. Improvements
*High Priority (EAP 6.0)*
1. *Ensure AS 7 compatibility.* See issue https://issues.jboss.org/browse/JBTM-905 JBTM-905.
2. *Move into the AS test suite*. ( https://issues.jboss.org/browse/JBQA-5192 https://issues.jboss.org/browse/JBQA-5192).
*Medium Priority (Post EAP 6.0)*
1. *Automate Emma and collate results*.
*Low Priority (Post EAP 6.0)*
1. *Update to use Arquillian*.
2. *Migrate to Maven*.
h2. Recovery Tests
The recovery tests are the tricky ones. At the moment we have a set of test scenarios that run in a single JBoss server invoked by a remote client. There are also a set of Byteman scripts that trigger failure at certain points in a scenario. There are many different permutations of scenario and Byteman scripts that each create a particular test.
The test scenarios log their progress through the protocol and recovery. On completion of a test run, a human needs to look over the trace and check that it looks right. This process is hard to automate as the trace produced can have many valid interleavings. This is due to the asynchronous nature of the application.
These tests are automated with a bash script, but the output traces must be verified manually by a human. The other problem is that they currently run in a single JBoss server which simulates the situation where every party in the protocol is located in the same app server and crashes & recovers at the same time (not that realistic!). In order to test with multiple servers. we would need a way of combining the traces from each server when verifying the outcome of the test. This could be done by implementing a Byteman helper class, but it would not be trivial.
h3. Improvements
*High Priority (EAP 6.0)*
1. *Move into AS test suite*. The tests are unlikely to be accepted in their current form, due to the level of manual intervention required. This improvement can not be made until we have sufficiently automated the process.
2. *Ensure AS 7 compatibility.* List issues here...
3. *Arquillian support*. Use Arquillian to automate the tests and hopefully remove the human verification step. https://issues.jboss.org/browse/JBQA-3926 https://issues.jboss.org/browse/JBQA-3926
*Medium priority (Post EAP 6.0)*
1. *Automate Emma and collate results*.
*Low Priority (Post EAP 6.0)
*
1. *Migrate to Maven*
2. *Multiple Server Tests*. If step 2) is successful, we could be able to build the more complex scenarios where each party runs in its own JBoss server.
3. *Additional Tests*. Andrew Dinn has provided a set of additional tests for us to consider implementing. These should be considered alongside the Emma coverage data for future work.
h2.
h2. Notes
* Arquillian supports multiple JBoss servers:* https://docs.jboss.org/author/display/ARQ/Multiple+Containers https://docs.jboss.org/author/display/ARQ/Multiple+Containers
* Arquillian doesn't yet support servers that crash. This feature is unlikely to make it into EAP 6.* https://issues.jboss.org/browse/ARQ-336 https://issues.jboss.org/browse/ARQ-336
--------------------------------------------------------------
Comment by going to Community
[http://community.jboss.org/docs/DOC-17242]
Create a new document in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=102&co...]
14 years, 3 months