[jboss-jira] [JBoss JIRA] (JGRP-1739) Test output leaking across test cases

[Richard Achmatowicz (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922546#comment-12922546 ] 

Richard Achmatowicz commented on JGRP-1739:
-------------------------------------------

It appears that in JUnitXMLReporter, the per-testcase sysout and syserr files are managed as follows:
For test cases:
* opened at the start of each test case (onTestStart)
* closed at the finish of each test case by intercepting the methods (onTestSuccess, onTestFailure, onTestSkipped)

For configurations:
* opened at the start of each configuration (beforeConfiguration)
* closed when a configuration fails (onConfigurationFailure)

In the case of successful configuration steps, this means that the files are left open, and someone else will have to close them.
Because configuration methods can occur after the test case has completed (@AfterClass, @AfterMethod), this means that in test case X, we can open sysout and syserr, and then will not get closed until the end of the first test method in the following test case. This seems to be what is happening.

  

                
> Test output leaking across test cases
> -------------------------------------
>
>                 Key: JGRP-1739
>                 URL: https://issues.jboss.org/browse/JGRP-1739
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 3.5
>            Reporter: Richard Achmatowicz
>            Assignee: Bela Ban
>            Priority: Minor
>
> Test case output written to System.out and System.err can be written from test case Y and end up in the output files of test case X, where Y is the test case immediately following X in the execution sequence.
> Here is an example obrained from running the ant target stack-independent::
> {noformat}
> stack-independent:
>     [mkdir] Created dir: /home/nrla/projects/jgroups-git-repo/tmp/test-results/xml/stack-independent
>      [xslt] Processing /home/nrla/projects/jgroups-git-repo/conf/testng/stack-independent.xml to /home/nrla/projects/jgroups-git-repo/tmp/stack-independent.xml
>      [xslt] Loading stylesheet /home/nrla/projects/jgroups-git-repo/conf/testng/parallelConfig.xsl
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testConcurrentMergeMultiplePartitions()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testConcurrentMergeMultiplePartitionsWithFlush()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testConcurrentMergeTwoPartitions()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testConcurrentMergeTwoPartitionsWithFlush()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testMergeAsymmetricPartitions()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testMergeAsymmetricPartitions2()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testMergeAsymmetricPartitionsWithFlush()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testMergeAsymmetricPartitionsWithFlush2()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testMergeRequestTimeout()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testMergeRequestTimeoutWithFlush()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testSimpleMerge()
>    [testng] OK:   [8] org.jgroups.protocols.GMS_MergeTest.testSimpleMergeWithFlush()
>    [testng] OK:   [8] org.jgroups.protocols.UNICAST_ContentionTest.testMessageReceptionUnderHighLoad-String()
>    [testng] OK:   [8] org.jgroups.protocols.UNICAST_ContentionTest.testMessageReceptionUnderHighLoad-String()
>    [testng] OK:   [8] org.jgroups.protocols.UNICAST_ContentionTest.testSimpleMessageReception-String()
>    [testng] OK:   [8] org.jgroups.protocols.UNICAST_ContentionTest.testSimpleMessageReception-String()
>    [testng] OK:   [8] org.jgroups.tests.TUNNELDeadLockTest.testStress()
>    [testng] OK:   [8] org.jgroups.tests.ConnectStressTest.testConcurrentJoining()
>    [testng] OK:   [8] org.jgroups.tests.SequencerOrderTest.testBroadcastSequence()
>    [testng] OK:   [8] org.jgroups.tests.SendAndReceiveTest.testSendAndReceiveWithDefaultUDP_Loopback()
>    [testng] OK:   [8] org.jgroups.tests.SendAndReceiveTest.testSendAndReceiveWithDefaultUDP_NoLoopback()
>    [testng] OK:   [8] org.jgroups.tests.SendAndReceiveTest.testSendAndReceiveWithLoopback()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testGeneratePreSignedUrlForDelete()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testGeneratePreSignedUrlForPut()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testGenerateQueryStringAuthenticationWithBasicGet()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testGenerateQueryStringAuthenticationWithBasicPost()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testGenerateQueryStringAuthenticationWithBasicPutAndHeaders()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testUsingPreSignedUrlWhenNotSet()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testUsingPreSignedUrlWhenSet()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testValidatePropertiesWithBothPreSignedSetButNoBucket()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testValidatePropertiesWithBothPreSignedSetButNoFile()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testValidatePropertiesWithBothPreSignedSetButTooManySubdirectories()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testValidatePropertiesWithBothPreSignedSetToValid()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testValidatePropertiesWithBothPreSignedSetToValidSubdirectory()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testValidatePropertiesWithPreSignedDeleteSet()
>    [testng] OK:   [8] org.jgroups.protocols.S3_PINGTest.testValidatePropertiesWithPreSignedPutSet()
>    [testng] OK:   [8] org.jgroups.tests.TCPGOSSIP_Test.testAddInitialHosts()
>    [testng] OK:   [8] org.jgroups.tests.TCPGOSSIP_Test.testConnectThree()
>    [testng] OK:   [8] org.jgroups.tests.TCPGOSSIP_Test.testConnectThreeChannelsWithGRAlreadyDown()
>    [testng] OK:   [8] org.jgroups.tests.TCPGOSSIP_Test.testConnectThreeChannelsWithGRDown()
>    [testng] OK:   [8] org.jgroups.tests.TCPGOSSIP_Test.testDisconnectConnectTwo()
>    [testng] OK:   [8] org.jgroups.tests.GossipRouterTest.testLateStart()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testConnectThree()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testDisconnectConnectOne_Default()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testDisconnectConnectOne_TUNNEL()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testDisconnectConnectSendTwo_Default()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testDisconnectConnectSendTwo_TUNNEL()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testDisconnectConnectTwo_Default()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testDisconnectConnectTwo_TUNNEL()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testFailureDetection()
>    [testng] OK:   [8] org.jgroups.tests.TUNNEL_Test.testNullLocalAddress_TUNNEL()
>    [testng] OK:   [8] org.jgroups.tests.SequencerMergeTest.testMergeAndSendOrdering()
>    [testng] OK:   [8] org.jgroups.tests.SequencerMergeTest.testMergeWithParticipant()
> BUILD SUCCESSFUL
> Total time: 4 minutes 20 seconds
> [1]+  Done                    emacs
> [nrla at lenovo jgroups-git-repo]$ grep -R testGeneratePreSignedUrlForDelete *
> Binary file classes/org/jgroups/protocols/S3_PINGTest.class matches 
> tests/junit/org/jgroups/protocols/S3_PINGTest.java:    public void testGeneratePreSignedUrlForDelete() {
> tmp/test-results/xml/stack-independent/org.jgroups.tests.SendAndReceiveTest/stdout.txt:------------- testGeneratePreSignedUrlForDelete -----------
> tmp/test-results/xml/stack-independent/TESTS-org.jgroups.tests.SendAndReceiveTest-stack-independent.xml:------------- testGeneratePreSignedUrlForDelete -----------
> Binary file tmp/test-results/xml/stack-independent/org.jgroups.protocols.S3_PINGTest/tests.data matches
> tmp/test-results/xml/stack-independent/TESTS-org.jgroups.protocols.S3_PINGTest-stack-independent.xml:    <testcase classname="org.jgroups.protocols.S3_PINGTest"  name="testGeneratePreSignedUrlForDelete" time="0.16">
> {noformat}
>  
> You'll see that some output from S3_PINGTest is written to the output file of SendAndReceiveTest, which precedes it in execution order. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira