Stefano Maestri [
http://community.jboss.org/people/maeste] created the discussion
"TestSuite - redesign of some tests and "false" coverage"
To view the discussion, visit:
http://community.jboss.org/message/573339#573339
--------------------------------------------------------------
Hi all,
A deep look at our test suite reveal that some of the coverage report, is not saying the
true about test situation.
Coverage is sometime overestimated. To say that better, cobertura is not wrong, from a
pure coverage point of view, but some of our tests (for examples parsers' ones) are
not well defined in terms of atomicity of behaviors verified. In a lot of cases, to
achieve our very short time release goal, we have written a generic tests stressing all
possible problems in just one tests. For example I've written my self this horrible
test:
@Test
public void shouldParseAnyExample() throws Exception
{
FileInputStream is = null;
//given
File directory = new
File(Thread.currentThread().getContextClassLoader().getResource("ds").toURI());
for (File xmlFile : directory.listFiles(new FileSuffixFilter("-ds.xml")))
{
System.out.println(xmlFile.getName());
try
{
is = new FileInputStream(xmlFile);
DsParser parser = new DsParser();
//when
DataSources ds = parser.parse(is);
//then
assertThat(ds.getDataSource().size() + ds.getXaDataSource().size(), is(1));
}
finally
{
if (is != null)
is.close();
}
}
}
@Test
public void shouldParseAnyExample() throws Exception
{
FileInputStream is = null;
//given
File directory = new
File(Thread.currentThread().getContextClassLoader().getResource("ds").toURI());
for (File xmlFile : directory.listFiles(new FileSuffixFilter("-ds.xml")))
{
System.out.println(xmlFile.getName());
try
{
is = new FileInputStream(xmlFile);
DsParser parser = new DsParser();
//when
DataSources ds = parser.parse(is);
//then
assertThat(ds.getDataSource().size() + ds.getXaDataSource().size(), is(1));
}
finally
{
if (is != null)
is.close();
}
}
}
Of course it give us a good coverage parsing a lot of file, bit in case of failure the
reasons for that failure could become a pain to understand and of course it doesn't
help a lot as regression tests.
I think we should review this kind of code to have more proper UNIT test :)
From this point of view I'd like very much also to limit as much
as possible xml files as test resources, defining xml to be parsed inline inside the tests
(for this kind of work scala language and its xml type would be a dream). Of course this
approach has the plus of having a real unit test with a fail containing really everything
to read and understand given-when-then conditions. The minus of this approach is that we
would provide less example files in our tests suite.
Please let me know what you
think about. My ideal solution would be to keep both approach, keeping the above test to
parse and use real xml file put in an integration-test phase (no coverage consideration
there) and the PURE UNIT tests described with inline xml file genration.
best regards
S.
--------------------------------------------------------------
Reply to this message by going to Community
[
http://community.jboss.org/message/573339#573339]
Start a new discussion in IronJacamar Development at Community
[
http://community.jboss.org/choose-container!input.jspa?contentType=1&...]