[jbossws-dev] JBossWS testsuite reorganization

[Rebecca Searls]

Apache'a Cargo api is worth investigating.  In a past life I evaluate
it for use in a different env.  I found it easy to use and flexible but not
quite what that other env needed.  I think it would be worth evaluating for 
our env.      

http://cargo.codehaus.org/
  Cargo is a thin wrapper that allows you to manipulate various type of 
  application containers (Java EE and others) in a standard way.  It has
  a Java API to start/stop/configure supported containers and ant tasks,
  Maven2/Maven3 plugins for configuring, starting, stopping and deploying 
  applications to supported containers.



----- Original Message -----
> From: "Alessio Soldano" <asoldano at redhat.com>
> To: jbossws-dev at lists.jboss.org
> Sent: Friday, June 13, 2014 11:39:24 AM
> Subject: [jbossws-dev] JBossWS testsuite reorganization
> 
> Folks,
> with current development on jbossws trunk aimed at version 5, I believe
> it's time consider rethinking the testsuite a bit.
> 
> Current status
> --------------------
> Historically, we've been running the integration tests against a JBoss
> AS container that has to be started before the testsuite. The target
> container is a vanilla distribution that suits with most of the tests,
> except for few ones requiring a https connector or a specific security
> domain to be available. Implementations of the
> org.jboss.wsf.spi.deployer.Deployer interface are available for each
> target container and allow deploying test archives as well as setting
> security domains / https connector by internally relying on the
> container management API.
> Different maven profiles trigger different implementations of Deployer
> (due to different artifact dependencies / classpath), allowing running
> the testsuite the same way against different target containers. When it
> comes to continuous integration, the idea is that a QA job first updates
> a target container with the current jbossws libraries, then it starts
> the container and runs the jbossws testsuite against it.
> 
> Issues / reasons for changing
> ---------------------------------------
> The current approach is working pretty much fine, but suffers from some
> limitations:
> * it needs a container to be already running when the integration test
> is triggered; it would be great if we could have the container started /
> stopped by the test itself
> * we're limited to running the testsuite against a predefined container
> profile (standalone.xml, standalone-full.xml ): while most of the tests
> makes sense when run against standalone.xml profile (because the
> webservices subsystem is on in it), we have some tests requiring
> subsystems available on different profiles (for instance, the messaging
> subsystem which is in standalone-full.xml or the xts subsystem which is
> in standalone-xts.xml). Currently we run the testsuite in Spring mode
> against standalone-full.xml, basically because till CXF 2.7.x the
> SOAP-over-JMS implementation has been available with Spring only. That's
> not the case anymore, on CXF 3.0.0, so we'd likely have to test it in
> both spring and non-spring mode, but that needs the messaging subsystem
> which is not in standalone.xml
> * we have few tests (but might have more in future) that actually
> require further additional changes to the server; that includes adding
> users, messaging queues, setting server side system properties at boot,
> ... The only way to achieve that atm is to have the QA job run a CLI
> script to modify the server or set a system property on command line
> when starting the server jvm
> * we have tests affecting the server management model, for instance
> changing the way the webservices subsystem model rewrites wsdl
> soap:address attributes; that makes it impossible to run those tests at
> the same time of any other tests
> 
> Proposals / idea
> ---------------------
> I'm listening to proposals to revisit the way the testsuite is run and
> better address the point above.
> Something I've been thinking about is relying on Arquillian, similarly
> to what the WFLY testsuite does. What's not trivial is the fact that we
> still need to have different version of the Arquillian SPI
> implementation (that is, different WFLY / EAP versions) used depending
> on a maven profile specified when running the testsuite; moreover we
> also need to be able to run against the result of building the latest
> WFLY master (iow, not a released target for which a
> wildfly-arquillian-container-xyz is available on the repository). We
> could have custom server profile files (standalone-abc.xml) in the
> jbossws sources for the supported containers to be used for starting
> specific versions of the containers through Arquillian. The integration
> tests could be split into groups meant to run against the various
> different profiles of the selected target container. Pretty much
> anything we need to customize on the server that can't be done using the
> Deployer approach can actually be done through a proper profile file (I
> would still keep the Deployer approach when working to limit the number
> of profile files to maintain, as there's a lot of stuff in there).
> Actually, we could even think about having a prepare phase before the
> actual testsuite that runs given CLI scripts to obtain the desired
> profile configuration.
> 
> Next steps
> ---------------
> Please provide feedback / ideas. Then we need a plan and eventually
> jiras created.
> 
> Cheers
> Alessio
> 
> --
> Alessio Soldano
> Web Service Lead, JBoss
> 
> _______________________________________________
> jbossws-dev mailing list
> jbossws-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbossws-dev
>