[jbossws-dev] JBossWS testsuite reorganization

[Alessio Soldano]

Hi Rostislav,

On 13/06/14 21:21, Rostislav Svoboda wrote:
>> * we have few tests (but might have more in future) that actually
>> require further additional changes to the server; that includes adding
>> users, messaging queues, setting server side system properties at boot,
>> ... The only way to achieve that atm is to have the QA job run a CLI
>> script to modify the server or set a system property on command line
>> when starting the server jvm
> ARQ helps here, you will have to switch default start/stop behavior to apply on every TestCase
>
>> * we have tests affecting the server management model, for instance
>> changing the way the webservices subsystem model rewrites wsdl
>> soap:address attributes; that makes it impossible to run those tests at
>> the same time of any other tests
> I thing this still applies even with ARQ usage - server is running all the time, it's not started for every test.
> So such tests should be isolated in separate execution or submodule and parallel test execution disabled.
Basically, I'm thinking about solving this the same as for the previous 
item (start a container for each of these tests and stop it when the 
test is over).


> * BOM file
>
> I would like to see definition of versions in BOM file and not in parent.
> This could help me a lot to run JBossWS TS with productized bits.
> Now we have to prepare patches for JBossWS codebase to have productized bits on client side too.
OK, this is however an independent task (a general project 
build/structure change), isn't it?


> * ARQ direct usage
>
> Still confused about dropping/not dropping old way and moving fully to ARQ.
> Have feeling that this proposal is on the half way to the full usage of ARQ.
> I think it's quite safe approach, you still have fallback plan.
Not sure what you mean here above; in any case, the move to use ARQ is 
still something being evaluated, not set in stone (yet).


> * integration tests using surefire or failsafe
>
> Wouldn't be failsafe more appropriate for integration tests ?
>    http://maven.apache.org/surefire/maven-failsafe-plugin/
>    The Failsafe Plugin is designed to run integration tests while the Surefire Plugins is designed to run unit tests.
Possibly yes; to be honest, though, given we don't run anything after 
the integration-test phase atm, there would be no change. But you're 
right, failsafe is probably more appropriate.


>
>> on a maven profile specified when running the testsuite; moreover we
>> also need to be able to run against the result of building the latest
>> WFLY master (iow, not a released target for which a
>> wildfly-arquillian-container-xyz is available on the repository).
> So first build latest WF to have latest wildfly-arquillian-container-xyz ?
Right, I didn't notice those artifacts actually come the WFLY build. So 
no problem here.


>> We could have custom server profile files (standalone-abc.xml) in the
>> jbossws sources for the supported containers to be used for starting
>> specific versions of the containers through Arquillian. The integration
> Different "configurations" will be definitely necessary (meaning arquillian.xml files), some things can be provided by parameters, but not sure if everything would be configurable this way.
> About standalone-abc.xml files - why would you have them ? Maintaining them isn't fun. I hope everything could be configurable using ARQ from default profiles/configurations, no custom confing files.
I share your concern; that's why I said I'd probably try something like 
a preliminary phase in which an existing server config file is copied, 
the server started, some cli commands sent to modify the config, the 
server stopped and eventually the server config referenced and used by 
arquillian. Maintaining standalone-abc.xml files should be the last option.


>
>> tests could be split into groups meant to run against the various
>> different profiles of the selected target container. Pretty much
>> anything we need to customize on the server that can't be done using the
>> Deployer approach can actually be done through a proper profile file (I
>> would still keep the Deployer approach when working to limit the number
>> of profile files to maintain, as there's a lot of stuff in there).
> Meaning to let ARQ to start WF/EAP with standalone.xml and execute current tests with prepared test libs?
>    ==> in the beginning you won't need to rewrite test archives using ShrinkWrap ?
I was actually still reasoning on the stanalone-abc.xml thing above. 
However, the migration to SW is something we wanted to do anyway 
(there's a jra for it). What you propose could be a interesting idea, 
the migration of ~600 tests to SW is not going to be a few hours tasks ;-)

Thanks
Alessio

-- 
Alessio Soldano
Web Service Lead, JBoss