[richfaces-issues] [JBoss JIRA] Created: (RF-10997) @Test annotation param dependsOnMethods in Metamer's ftest causes IllegalStateException

[Jan Jamrich (JIRA)]

@Test annotation param dependsOnMethods in Metamer's ftest causes IllegalStateException
---------------------------------------------------------------------------------------

                 Key: RF-10997
                 URL: https://issues.jboss.org/browse/RF-10997
             Project: RichFaces
          Issue Type: Bug
      Security Level: Public (Everyone can see)
         Environment: Metamer 4.1.0-snapshot, r.22501
            Reporter: Jan Jamrich


When tried set dependsOnMethods param to @Test annotation in Metamer's ftests, get IllegalStateException thrown from MatrixConfigurator.

There aren't any other methods annotated by @Test with dependsOnMethods param, so its possible that this issue is in Metamer longer time without being discovered.

By debugging observed that list of methods set to be executed by TestRunner depends on this param. In other word, when set dependsOnMethods param to @Test annotation - this method (on which was this param set to annotation) doesn't appear in list of methods being executed (LinkedList<Method> methods attribute of MatrixConfigurator).
This attribute is initialized in method 
public List<IMethodInstance> intercept(List<IMethodInstance> methodInstances, ITestContext context)
which is called directly from org.testng.TestRunner, which give param List<IMethodInstance> methodInstances depending on dependsOnMethods param set to @Test annotation as well.

This behavior should be ok, since methods which are not depending on another else method should be executed first.
But what seems worse is that when run this again - for method anotated by @Test with param dependsOnMethod, then list of methods which going to be executed is empty - do no test method called.

And in case when all methods except one depends on another method, different exception is result of run testsuite:
Running TestSuite
org.apache.maven.surefire.booter.SurefireExecutionException: null; nested exception is java.lang.NullPointerException: null
java.lang.NullPointerException
	at org.richfaces.tests.metamer.ftest.MetamerTestInfo.getConfigurationInfo(MetamerTestInfo.java:50)
	at org.richfaces.tests.metamer.ftest.MetamerTestInfo.getConfigurationInfoInParenthesses(MetamerTestInfo.java:77)
	at org.richfaces.tests.metamer.ftest.MetamerSeleniumLoggingTestListener.getMessage(MetamerSeleniumLoggingTestListener.java:35)
	at org.jboss.test.selenium.listener.SeleniumLoggingTestListener.logStatus(SeleniumLoggingTestListener.java:82)
	at org.jboss.test.selenium.listener.SeleniumLoggingTestListener.onTestFailure(SeleniumLoggingTestListener.java:55)
	at org.testng.internal.Invoker.runTestListeners(Invoker.java:1634)
	at org.testng.internal.Invoker.runTestListeners(Invoker.java:1618)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1101)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:137)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:121)
	at org.testng.TestRunner.runWorkers(TestRunner.java:1030)
	at org.testng.TestRunner.privateRun(TestRunner.java:709)
	at org.testng.TestRunner.run(TestRunner.java:579)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:331)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288)
	at org.testng.SuiteRunner.run(SuiteRunner.java:193)
	at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:879)
	at org.testng.TestNG.run(TestNG.java:787)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
	at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
	at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)


So this should be issue related to TestNG itself (current version 5.12.1), tried 5.13, but the same behavior.


There is stack trace:
java.lang.IllegalStateException: can't find more configured methods
	at org.richfaces.tests.metamer.ftest.MatrixConfigurator.configureMethod(MatrixConfigurator.java:150)
	at org.richfaces.tests.metamer.ftest.MatrixConfigurator.beforeInvocation(MatrixConfigurator.java:101)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
	at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
	at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
	at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)
... Removed 22 stack frames

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira