@Test annotation param dependsOnMethods in Metamer's ftest causes IllegalStateException --------------------------------------------------------------------------------------- Key: RFPL-1467 URL: https://issues.jboss.org/browse/RFPL-1467 Project: RichFaces Planning Issue Type: Bug Security Level: Public(Everyone can see) Components: QE Environment: Metamer 4.1.0-snapshot, r.22501 Reporter: Jan Jamrich When tried set dependsOnMethods param to @Test annotation in Metamer's ftests, get IllegalStateException thrown from MatrixConfigurator. There aren't any other methods annotated by @Test with dependsOnMethods param, so its possible that this issue is in Metamer longer time without being discovered. By debugging observed that list of methods set to be executed by TestRunner depends on this param. In other word, when set dependsOnMethods param to @Test annotation - this method (on which was this param set to annotation) doesn't appear in list of methods being executed (LinkedList<Method> methods attribute of MatrixConfigurator). This attribute is initialized in method public List<IMethodInstance> intercept(List<IMethodInstance> methodInstances, ITestContext context) which is called directly from org.testng.TestRunner, which give param List<IMethodInstance> methodInstances depending on dependsOnMethods param set to @Test annotation as well. This behavior should be ok, since methods which are not depending on another else method should be executed first. But what seems worse is that when run this again - for method anotated by @Test with param dependsOnMethod, then list of methods which going to be executed is empty - do no test method called. And in case when all methods except one depends on another method, different exception is result of run testsuite: Running TestSuite org.apache.maven.surefire.booter.SurefireExecutionException: null; nested exception is java.lang.NullPointerException: null java.lang.NullPointerException at org.richfaces.tests.metamer.ftest.MetamerTestInfo.getConfigurationInfo(MetamerTestInfo.java:50) at org.richfaces.tests.metamer.ftest.MetamerTestInfo.getConfigurationInfoInParenthesses(MetamerTestInfo.java:77) at org.richfaces.tests.metamer.ftest.MetamerSeleniumLoggingTestListener.getMessage(MetamerSeleniumLoggingTestListener.java:35) at org.jboss.test.selenium.listener.SeleniumLoggingTestListener.logStatus(SeleniumLoggingTestListener.java:82) at org.jboss.test.selenium.listener.SeleniumLoggingTestListener.onTestFailure(SeleniumLoggingTestListener.java:55) at org.testng.internal.Invoker.runTestListeners(Invoker.java:1634) at org.testng.internal.Invoker.runTestListeners(Invoker.java:1618) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1101) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:137) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:121) at org.testng.TestRunner.runWorkers(TestRunner.java:1030) at org.testng.TestRunner.privateRun(TestRunner.java:709) at org.testng.TestRunner.run(TestRunner.java:579) at org.testng.SuiteRunner.runTest(SuiteRunner.java:331) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288) at org.testng.SuiteRunner.run(SuiteRunner.java:193) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910) at org.testng.TestNG.runSuitesLocally(TestNG.java:879) at org.testng.TestNG.run(TestNG.java:787) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141) at org.apache.maven.surefire.Surefire.run(Surefire.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009) So this should be issue related to TestNG itself (current version 5.12.1), tried 5.13, but the same behavior. There is stack trace: java.lang.IllegalStateException: can't find more configured methods at org.richfaces.tests.metamer.ftest.MatrixConfigurator.configureMethod(MatrixConfigurator.java:150) at org.richfaces.tests.metamer.ftest.MatrixConfigurator.beforeInvocation(MatrixConfigurator.java:101) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141) at org.apache.maven.surefire.Surefire.run(Surefire.java:177) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009) ... Removed 22 stack frames