[jbosstools-dev] AS Test Suite failure fix?

[Max Rydahl Andersen]

On Tue, Mar 19, 2013 at 10:33:37PM -0400, Nick Boldt wrote:
>Does it strike no one as a problem that the tests pass against Juno SR2
>but fail against Juno R? Surely they should pass against any Juno on
>which we run the tests, since we can't guarantee that a user will run on
>a particular version.

The bug found a behaviorial change in WTP 3.3.0 to 3.3.1.

This change is for the better (removes any weird eclipse only artifacts in deployments),
but has the big problem of that we will see different behaviors where we run.

But this is kinda expected since a micro release is a place where bugfixes are done and
thus you will see different behaviors.

To make the test pass on both versions the test will need to lookup which bundle version it
us running in - but then you'll have this test pass on SR0 even though SR0 actually have this
bug which was fixed in SR1+.

I think the best thing is that these or at least one of tests have their message updated to say something like:
assertEquals(".project files should not be included. Broken in SR0, fixed in SR1", ...) or similar allowing
to actually see it when tests fails what/why this changed.

>Or will we document this like we did for JBT 3.3 [1], to at least
>ENCOURAGE users to use SR2 instead of R or SR1?

Yes, we should always do that anyway (i.e. recommend to use the latest Eclipse we've tested against)

What is/was bad aobut this whole case is that it took so long to realize since this issue
must have been failing for months on SR0 :(

/max

>[1]
>https://www.jboss.org/tools/download/installation/update_3_3#installon370371
>
>On 03/19/2013 02:27 PM, Denis Golovin wrote:
>> Hudson is actually running build agaains minimum TP, because it has no
>> -Dmaximum switch in maven command line.
>>
>> Denis
>>
>> On 03/19/2013 02:10 AM, Rob Stryker wrote:
>>
>>> https://github.com/jbosstools/jbosstools-server/commit/b3672b46e4df710741c39ff77733246485547c8c
>>>
>>>
>>> Denis:
>>>
>>> I realize the error in the test suite was that the .project file is
>>> now included, whereas it was not included in the past.  Why do you
>>> think the proper solution to this change in behavior was to simply
>>> change the expected count? That seems like a very bad fix.
>>>
>>> If something upstream has changed, we should discover what it is
>>> before simply changing the expected count in the unit test. Clearly
>>> these tests used to pass, and started failing after the update to m5.
>>> Shouldn't we try to discover what exactly changed here?
>>>
>>> - Rob
>>
>> _______________________________________________
>> jbosstools-dev mailing list
>> jbosstools-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>
>
>-- 
>Nick Boldt :: JBoss by Red Hat
>Productization Lead :: JBoss Tools & Dev Studio
>http://nick.divbyzero.com
>_______________________________________________
>jbosstools-dev mailing list
>jbosstools-dev at lists.jboss.org
>https://lists.jboss.org/mailman/listinfo/jbosstools-dev