[jbosstools-issues] [JBoss JIRA] (JBIDE-19081) Use simpler Surefire include/exclude pattern in parent pom

[Nick Boldt (JIRA)]

    [ https://issues.jboss.org/browse/JBIDE-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261955#comment-13261955 ] 

Nick Boldt commented on JBIDE-19081:
------------------------------------

Here's a quick way to parse locally generated surefire test logs for a count of tests run per test, then count up the total number of classes and tests run, too. 

I've also parsed out numbers for Failures, Errors and Skips, so you can track if test statuses change.

For example, run this in the folder where you have jbosstools-openshift checked out:

{code}
echo "Total, Run, Failures, Errors, Skips :: path/to/surefire-reports/class-file.txt"
num=0; tot=0; tests=$(find */*/*/target/surefire-reports/ */*/target/surefire-reports/ -mindepth 1 -maxdepth 1 -name "*.txt" 2>/dev/null|sort); for d in $tests; do (( num++ )); cnt=""; cnt=$(cat $d | grep -v "Tests run" | grep "Time elapsed" | wc -l); tot=$(( tot + cnt )); cat $d | grep "Tests run" | sed "s#Tests run: \([0-9]\+\), Failures: \([0-9]\+\), Errors: \([0-9]\+\), Skipped: \([0-9]\+\), Time elapsed: .\+ in \(.\+\).*#\1 ${cnt} \2 \3 \4 :: ${d}#"; done; echo ""; echo "Total classes: $num"; echo "Total tests run: $tot"

{code}
Output looks like this:
{code}
Total, Run, Failures, Errors, Skips :: path/to/surefire-reports/class-file.txt
3 3 0 0 0 :: tests/org.jboss.tools.openshift.cdk.server.test/target/surefire-reports/org.jboss.tools.openshift.cdk.server.test.internal.CDKDockerUtilityTest.txt
1 1 0 0 0 :: tests/org.jboss.tools.openshift.cdk.server.test/target/surefire-reports/org.jboss.tools.openshift.cdk.server.test.internal.CDKLaunchControllerTest.txt
...
4 3 0 0 1 :: tests/org.jboss.tools.openshift.test/target/surefire-reports/org.jboss.tools.openshift.test.core.connection.ConnectionPersistencyTest.txt
...
1 0 0 0 1 :: tests/org.jboss.tools.openshift.test/target/surefire-reports/org.jboss.tools.openshift.test.ui.wizard.deployimage.search.DockerHubRegistryTest.txt
...
4 4 0 0 0 :: tests/org.jboss.tools.openshift.test/target/surefire-reports/org.jboss.tools.openshift.test.ui.wizard.newapp.TemplateParameterViewerUtilsTest.txt

Total classes: 93
Total tests run: 626

{code}

or
{code}
Total, Run, Failures, Errors, Skips :: path/to/surefire-reports/class-file.txt
12 12 0 0 0 :: common/tests/org.jboss.tools.common.core.test/target/surefire-reports/org.jboss.tools.common.core.test.CommonCoreTestSuite.txt
...
1 0 0 0 1 :: common/tests/org.jboss.tools.common.verification.ui.test/target/surefire-reports/org.jboss.tools.common.verification.ui.test.VerificationUiAllTests.txt
...
50 50 0 0 0 :: usage/tests/org.jboss.tools.usage.test/target/surefire-reports/org.jboss.tools.usage.test.UsageTestSuite.txt

Total classes: 21
Total tests run: 343
{code}

Then you can diff that will another test run to see if either a line is missing (test class didn't run) or the count is different (tests within the class didn't run).


> Use simpler Surefire include/exclude pattern in parent pom
> ----------------------------------------------------------
>
>                 Key: JBIDE-19081
>                 URL: https://issues.jboss.org/browse/JBIDE-19081
>             Project: Tools (JBoss Tools)
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 4.3.0.Alpha1
>            Reporter: Nick Boldt
>            Assignee: Nick Boldt
>            Priority: Minor
>             Fix For: 4.4.x
>
>
> 1. In JBDS9, use these new default patterns for Surefire to define which test classes to run/exclude:
> {code}
> include = *Test*, *Test, *TestCase
> exclude = *Abstract*
> {code}
> 2. If that causes test failures because running incorrectly named
> abstract stuff, they can refactor, add their own root pom overrides, use
> a TestSuite, or use @Ignore in test classes.
> 3. If the count of tests run suddenly DROPS because the pattern isn't
> running the correct # of tests, they can add their own root pom
> overrides, or use a TestSuite.
> Ref: http://lists.jboss.org/pipermail/jbosstools-dev/2015-January/009688.html



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)