[webbeans-dev] TCK - grouping broken etc. tests

Hi all, I just spent some time working through the tests marked as ri-broken tests in the TCK. I've fixed those I can easily, and filed WBRI JIRAs for the rest. It made me realize we need some guidelines on how to handle tests which don't work: http://www.seamframework.org/WebBeans/GroupingTests * If the test isn't working properly (e.g. you haven't finished it, you need to check it in to get advice from others, you find a test and can see it is broken, either due to changes in the spec, or due to a bad test in the first place, you need to add it to the "broken" group - @Test (groups={"broken"}). Please also add a note saying what is wrong with the test, and if you've spent some time with it, note where you got to with understanding/fixing it. Broken tests are highlighted in the audit in yellow. If you are working on a section, always check to see if you can't fix the broken tests. Alternatively, if you've written a test which will work standalone, but won't work in the container, add it to the incontainer-broken group * If you think the test is correct, but it won't run against the RI you need to: - add it to the ri-broken group - @Test groups={"broken"}) - alternatively, if the test will pass standalone, but won't pass in JBoss AS (e.g. we've had this where serialization is involved), then add it to the incontainer-ri-broken group - check and see if there is already a matching WBRI issue, if there isn't create one - add a note to the issue with the FQN of the test method (use copy qualified name in eclipse on the method) - add a comment to the test indicating the WBRI issue number * If you find a test that is correct, but could be more complete, add it to the rewrite group @Test(groups={"rewrite"}) * Do not use the stub group any more * Feel free to tag tests using groups to mark what they relate to. Eventually, I would like to go through and make a consistent effort at this with a well defined list of tags, but this is very low priority. * We will also use this approach to allow testing on an embeddable EJB 3.1 container. If you are tagging a test that will definitely run on this container (for example, a JSF test won't run, a servlet test won't run) then please add the ejb-embeddable tag. Again, this is low priority for now Pete