[hibernate-dev] updated testing stuff

[Steve Ebersole]

I wanted to give everyone a heads up about the changes to the stuff in 
org.hibernate.testing (there is still a question about whether Eclipse is able 
to handle the hibernate-testing as a separate module setup).

Internally, the main piece is a custom JUnit4 runner implementation, 
org.hibernate.testing.junit4.CustomRunner.  Some things it does:
1) The first major difference is that it creates a single instance of the test 
class for all the methods.  This is completely different than stock JUnit and 
different from what we used to have.
2) Applies around-advice for custom @BeforeClassOnce and @AfterClassOnce 
annotations
3) Applies around-advice handling for @FailureExpected, @Skip, 
@SkipForDialect, @RequiresDialect and @RequiresDialectFeature annotations.  
Another major difference from the previous incarnation of this code is that 
instead of not even simply dropping "ignored" tests from the testsuite, we now 
treat them as if they had been marked with JUnit's @Ignore.  This works nicely 
with JUnit reports and IDEs.

Some notes about the specific custom annotations:
1) @BforeClassOnce - much like JUnit's own @BeforeClass except that the 
annotated method need not be static.
2) @AfterClassOnce - much like JUnit's own @AfterClass except that the 
annotated method need not be static.
3) @FailureExpected - Marks a method (or at class-level to mark all method in 
class) as being a failure expected.  If 
'hibernate.test.validatefailureexpected' is defined as a System property as 
'true' then this behaves just like older "failure expected" handling; 
otherwise the test is treated as an ignore.
4) @OnFailure - annotates a method to be performed on test failure.
5) @OnExpectedFailure - annotates a method to be performed on an expected test 
failure.
6) @RequiresDialect (method or class) - The test will be run only if the 
executing dialect matches the specified dialect.
7) @RequiresDialect (method or class) - The test will be run only if the 
executing dialect matches the specified feature.
8) @Skip (method or class) - defines a general purpose, conditional skip.  The 
test is skipped if the specified condition is met.
9) @SkipForDialect (method or class) - The test will be run only if the 
executing dialect does not match the specified dialect.
10) @TestForIssue (method or class) - just an informational annotation for 
now.  Allows to specify the JIRA key for the issue the test demonstrates.

It is also worth noting that only methods explicitly marked with 
@org.junit.Test are considered test methods.  

---
Steve Ebersole <steve at hibernate.org>
http://hibernate.org