[jboss-dev-forums] [IronJacamar Development] - TestSuite - redesign of some tests and " false" coverage

Stefano Maestri [http://community.jboss.org/people/maeste] created the discussion "TestSuite - redesign of some tests and "false" coverage" To view the discussion, visit: http://community.jboss.org/message/573339#573339 -------------------------------------------------------------- Hi all, A deep look at our test suite reveal that some of the coverage report, is not saying the true about test situation. Coverage is sometime overestimated. To say that better, cobertura is not wrong, from a pure coverage point of view, but some of our tests (for examples parsers' ones) are not well defined in terms of atomicity of behaviors verified. In a lot of cases, to achieve our very short time release goal, we have written a generic tests stressing all possible problems in just one tests. For example I've written my self this horrible test: @Test    public void shouldParseAnyExample() throws Exception    {       FileInputStream is = null;       //given       File directory = new File(Thread.currentThread().getContextClassLoader().getResource("ds").toURI());       for (File xmlFile : directory.listFiles(new FileSuffixFilter("-ds.xml")))       {          System.out.println(xmlFile.getName());          try          {             is = new FileInputStream(xmlFile);             DsParser parser = new DsParser();             //when             DataSources ds = parser.parse(is);             //then             assertThat(ds.getDataSource().size() + ds.getXaDataSource().size(), is(1));          }          finally          {             if (is != null)                is.close();          }       }    }    @Test    public void shouldParseAnyExample() throws Exception    {       FileInputStream is = null;         //given       File directory = new File(Thread.currentThread().getContextClassLoader().getResource("ds").toURI());       for (File xmlFile : directory.listFiles(new FileSuffixFilter("-ds.xml")))       {          System.out.println(xmlFile.getName());          try          {             is = new FileInputStream(xmlFile);             DsParser parser = new DsParser();             //when             DataSources ds = parser.parse(is);             //then             assertThat(ds.getDataSource().size() + ds.getXaDataSource().size(), is(1));            }          finally          {             if (is != null)                is.close();          }       }    } Of course it give us a good coverage parsing a lot of file, bit in case of failure the reasons for that failure could become a pain to understand and of course it doesn't help a lot as regression tests. I think we should review this kind of code to have more proper UNIT test :) Please let me know what you think about. My ideal solution would be to keep both approach, keeping the above test to parse and use real xml file put in an integration-test phase (no coverage consideration there) and the PURE UNIT tests described with inline xml file genration. best regards S. -------------------------------------------------------------- Reply to this message by going to Community [http://community.jboss.org/message/573339#573339] Start a new discussion in IronJacamar Development at Community [http://community.jboss.org/choose-container!input.jspa?contentType=1&...]