JBossWS testsuite reorganization
by Alessio Soldano
Folks,
with current development on jbossws trunk aimed at version 5, I believe
it's time consider rethinking the testsuite a bit.
Current status
--------------------
Historically, we've been running the integration tests against a JBoss
AS container that has to be started before the testsuite. The target
container is a vanilla distribution that suits with most of the tests,
except for few ones requiring a https connector or a specific security
domain to be available. Implementations of the
org.jboss.wsf.spi.deployer.Deployer interface are available for each
target container and allow deploying test archives as well as setting
security domains / https connector by internally relying on the
container management API.
Different maven profiles trigger different implementations of Deployer
(due to different artifact dependencies / classpath), allowing running
the testsuite the same way against different target containers. When it
comes to continuous integration, the idea is that a QA job first updates
a target container with the current jbossws libraries, then it starts
the container and runs the jbossws testsuite against it.
Issues / reasons for changing
---------------------------------------
The current approach is working pretty much fine, but suffers from some
limitations:
* it needs a container to be already running when the integration test
is triggered; it would be great if we could have the container started /
stopped by the test itself
* we're limited to running the testsuite against a predefined container
profile (standalone.xml, standalone-full.xml ): while most of the tests
makes sense when run against standalone.xml profile (because the
webservices subsystem is on in it), we have some tests requiring
subsystems available on different profiles (for instance, the messaging
subsystem which is in standalone-full.xml or the xts subsystem which is
in standalone-xts.xml). Currently we run the testsuite in Spring mode
against standalone-full.xml, basically because till CXF 2.7.x the
SOAP-over-JMS implementation has been available with Spring only. That's
not the case anymore, on CXF 3.0.0, so we'd likely have to test it in
both spring and non-spring mode, but that needs the messaging subsystem
which is not in standalone.xml
* we have few tests (but might have more in future) that actually
require further additional changes to the server; that includes adding
users, messaging queues, setting server side system properties at boot,
... The only way to achieve that atm is to have the QA job run a CLI
script to modify the server or set a system property on command line
when starting the server jvm
* we have tests affecting the server management model, for instance
changing the way the webservices subsystem model rewrites wsdl
soap:address attributes; that makes it impossible to run those tests at
the same time of any other tests
Proposals / idea
---------------------
I'm listening to proposals to revisit the way the testsuite is run and
better address the point above.
Something I've been thinking about is relying on Arquillian, similarly
to what the WFLY testsuite does. What's not trivial is the fact that we
still need to have different version of the Arquillian SPI
implementation (that is, different WFLY / EAP versions) used depending
on a maven profile specified when running the testsuite; moreover we
also need to be able to run against the result of building the latest
WFLY master (iow, not a released target for which a
wildfly-arquillian-container-xyz is available on the repository). We
could have custom server profile files (standalone-abc.xml) in the
jbossws sources for the supported containers to be used for starting
specific versions of the containers through Arquillian. The integration
tests could be split into groups meant to run against the various
different profiles of the selected target container. Pretty much
anything we need to customize on the server that can't be done using the
Deployer approach can actually be done through a proper profile file (I
would still keep the Deployer approach when working to limit the number
of profile files to maintain, as there's a lot of stuff in there).
Actually, we could even think about having a prepare phase before the
actual testsuite that runs given CLI scripts to obtain the desired
profile configuration.
Next steps
---------------
Please provide feedback / ideas. Then we need a plan and eventually
jiras created.
Cheers
Alessio
--
Alessio Soldano
Web Service Lead, JBoss
9 years, 11 months
- 4
- 34
Fwd: EndpointMetrics Refactor
by Alessio Soldano
Moving discussion to public dev list
-------- Original Message --------
Subject: EndpointMetrics Refactor
Date: Tue, 01 Jul 2014 15:58:46 +0800
From: Jim Ma <ema(a)redhat.com>
To: Alessio Soldano <asoldano(a)redhat.com>, Rebecca Searls
<rsearls(a)redhat.com>
Hi Alessio & Rebecca,
After discussed on IRC last week, looked at cxf's management code in
more depth
and wrote something to verify yesterday, it's time to sum up what can we
do to refactor
the current EndpointMetrics as I promised to make it better work with
cxf's management
implementation.
At the beginning, I'd like to quickly list the cons of the current
EndpointMetrics
implementation.
- Because the statistics is fulfilled in RequestHandlerImpl , after the
response is successfully
returned to receiver. The response related metric value could not
be updated in time, more
details pleas see JBWS-3808
- It doesn't count the one way fault correctly.
- It doesn't expose all the attribute in jmx.
- If the cxf's management is enabled, user will get two different
statistics value. One is
calculated with jbossws-cxf's endpointmetrics which happens in
RequestHandler(servlet),
and another one is calculated with ResponseTimeInterceptor(cxf's
interceptor).
The pros of current implementation :
- It gives more accurate start time for each request.
- It is simple and doesn't need to add more interceptor/extra code in
the runtime to collect
the data.
From the above analysis, one important thing we need to do is make the
EndpintMetrics seamlessly
work with cxf's management intercetptor and get statistics from one
place instead of generate two
copies from different implementation.
There is CounterRepostiry extension in cxf which responsible for
creating ResponseTimeCounter
for each endpoint, It can be the only source we get statistics for each
endpoint.
To get metric value from this class, we need to change following things:
1. Each ResponseTImeCounter(something similar to EndpointMetrics) is
lazily created in
message interceptor and put in CounterReposiory with a long
ObjectName at the moment. We
need to change it and add the ResponseTimeCounter for each jbossws
endpoint , then put it in
CounterRepostiry with key name of endpoint name.
2. Change EndpointMetricsFactory abstract class to create with
EndpointMetric with endpoint
parameter so the endpoint name can be used to create
ResponseTimeCounter.
3. Create new EndpointMetricImpl in jbossws-cxf and get metric value
from ResponseCounter
which is created for each endpoint.
4. Change the ResponseTimeInterceptor and allow for define the counter
object name.
Besides this , I also need to refactor the ResponseTimeInterceptor to
make it log the process time
more accurate(by move ResponseTimeInInterceptor before several
interceptors ?). And also refactor
the ResponseTimeInterceptor a bit to handle the fault message correctly.
If you have any question and comments about this change , please let me
know.
Thanks,
Jim
10 years, 5 months
- 2
- 11
- ← Newer
- 1
- Older →