I don't think you can (or should) avoid test runs for the different runtimes (AS 4.2/5.0), no matter how much time they take. Otherwise how we know it works? If QA needs more machines, then so be it. The real issue is to have as much as possible a single codebase. JBossWS uses a common "core" and different integration layers depending on the runtime (as40, as42, as50). Tim Fox wrote: > A posted something similar on this some time ago, but we didn't get > to any conclusions so I'm bringing it up again. > > JBoss Messaging has all its dependencies aligned with JBAS 4.2.0 - > i.e. we use all the same versons of libraries as AS 4.2.0. > > This means all the testing we did is valid when we install inside AS > 4.2.0 since we know we're still using all the same libraries. > > However, JBAS 5.0 uses different versions of libraries to AS 4.2.0 - > so we can't guarantee that the version of our project that runs in > 4.2 will run in AS 5.0 - since we don't test against the versions of > the libraries used by AS 5.0. > > The only way around this AFAICT is for us to keep two versions of JBM > (or at least to have two sets of dependencie)- one with a set of > dependencies mirroring AS 4.2.0, and another with a set of > dependencies mirroring AS 5.0. > > But this is a big maintenance hassle - and will mean double the test > runs. Our test runs already take forever. > > How do other projects deal with this? Surely we are not the only ones. > > Or perhaps other projects don't worry about it and "hope for the > best" when installed in AS 5.0? > _______________________________________________ > jboss-development mailing list > jboss-development(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-development _______________________________________________ jboss-development mailing list jboss-development(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-development

On Thu, 2007-07-19 at 16:08 +0100, Tim Fox wrote: > The issue is really just that we test against version X of aop, version > Y of jgroups, version Z of core libs since that is what AS 4.2 uses, so > our tests are only valid with those versions. > You're talking about integration testing. That's not your problem, that's a problem for the AS integration testsuite. It's only your problem if your doing the integration of JBoss Messaging version X into JBossAS version Y. You run the integration tests and if they don't pass, you don't commit. What is your problem is those integration tests fail because JGroups version A doesn't work like JGroups version B because you used something that either wasn't meant to be used that way or that wasn't regression tested for the way you use it. It's only by solving the second problem (clean and tested integration between individual projects) that the first problem (JBossAS can pick and choose which versions of projects it wants to integrate) doesn't become an N^2 problem. You've still got to run two integration tests, jboss-4.2.x and jboss-5.x, regardless.

Mocking is gay and the worst idea I ever heard of. You end up just testing mock implementations.

AOP and commons are an API issue, not an SPI one. You're not gonna be able to mock AOP or commons even if you thought that was a good idea.

There's nothing to stop you delivering your tests into the AS as part of the integration. Same as what we do with the joram tests. Except of course you have your own test framework (which is something I complained about a while ago) instead of reusing and extending jboss-test, so that will be hard. On Thu, 2007-07-19 at 16:37 +0100, Tim Fox wrote: > So what you're saying is that a lot (maybe most) of the tests in our > test suite are actually integration tests and should go into AS. (The > current messaging integration tests in are really crappy) > > And the JBM test suite should only deal with mocks. So we need to mock > out usage of JGroups, AOP, the database, commons core etc > In principle, there should be no real difference between using mocks and picking a single version of those projects to test against. If the api is stable it should make no difference. In reverse, there should be a simple cutdown configuration of JBoss Messaging (no persistence, security, clustering, networking, etc.) that others can use in their tests as a "Mock JMS". You've really got two choices of version, both have their advantages and disadvantages. 1) Choose the same versions as the current stable branch of JBossAS, at time of writing JBoss-4.2 Advantage: Its easy for you to work out problems with the integration in that branch Disadvantage: Its harder to test integration with JBoss-5 or other branches. You have to debug in the JBossAS codebase instead of your own. 2) Always choose the lastest reasonably stable version of each project Advantage: You're catching regressions and problems in those projects early. Disadvantage(s): You could easily introduce a dependency on a new feature of those projects that isn't available in the version chosen by JBoss-4.2. And you're only going to find integration problems caused by old bugs in the other projects (fixed in the lastet versions) when you run the JBossAS integration tests. But that's really a problem for the other project. :-) In practice, you probably want to switch between these strategies depending upon where you are in your development lifecycle. i.e. strategy (2 - latest and "greatest") for alphas/betas and strategy (1 - more stable versions) as you get towards a more stable release yourself.

On Thu, 2007-07-19 at 17:48 +0300, Dimitris Andreadis wrote: > The real issue is to have as much as possible a single codebase. JBossWS uses a common > "core" and different integration layers depending on the runtime (as40, as42, as50). That is the wrong approach. It's not JBossWS's job to define how webservices integrate into JBossAS. JBossAS should define a stable integration point, test it and JBossWS should implement it.

I'm currently going through the process of fixing AOP (which takes the same wrong approach) to use the new jboss-classloading-spi instead of "deep diving" into the UnifiedClassLoader. For legacy reasons, it's difficult to fix the problem in 4.x (we can't change the api now it is released) but we can in JBoss5 and we can come up with something that is; 1) supportable for a long time. 2) makes it easier to understand what work needs to be done to integrate a project with other "platforms" - e.g. running JBoss Messaging inside Tomcat, Eclipse or a unit test, etc. Oh no Tim! More platforms/versions to test. :-)

+100 All integration code should live in AS. Otherwise it becomes nearly impossible to make large changes to AS, since such integration modules rely on internal implementation details. Not to mention it would be a royal pain to track down and fix each component, since they would be in separate repositories.

Adrian wrote: > On Thu, 2007-07-19 at 17:48 +0300, Dimitris Andreadis wrote: >> The real issue is to have as much as possible a single codebase. JBossWS uses a common >> "core" and different integration layers depending on the runtime (as40, as42, as50). > > That is the wrong approach. It's not JBossWS's job to define how > webservices integrate into JBossAS. > > JBossAS should define a stable integration point, test it and JBossWS > should implement it. > > I'm currently going through the process of fixing AOP > (which takes the same wrong approach) > to use the new jboss-classloading-spi instead of > "deep diving" into the UnifiedClassLoader. > > For legacy reasons, it's difficult to fix the problem in 4.x > (we can't change the api now it is released) > but we can in JBoss5 and we can come up with something > that is; > 1) supportable for a long time. > 2) makes it easier to understand what work > needs to be done to integrate a project with > other "platforms" - e.g. running JBoss Messaging inside Tomcat, > Eclipse or a unit test, etc. > > Oh no Tim! More platforms/versions to test. :-)

But this is a big maintenance hassle - and will mean double the test runs. Our test runs already take forever. How do other projects deal with this? Surely we are not the only ones.

Or perhaps other projects don't worry about it and "hope for the best" when installed in AS 5.0?