[jbosstools-dev] Test failures/timeout are very bad

[Max Rydahl Andersen]

On Tue, May 21, 2013 at 10:46:24AM +0200, Mickael Istria wrote:

Mickael - problem is that these tests passes fine on a local machine.

Fails on jenkins which unfortunately keeps being extremly unreliable and slow.

But Yes - we need to fix these and adjust the tests to not be reliant
on multiple jobs/threads etc. so the tests can run to completeness more reliable.

thats the worst thing I see in our testsuite, the tests are relying on concurrent jobs
and aren't simply testing a unit of code. it's testing a combination (i.e. big integration test)
and that's not something we should have in our core tests - thats too unreliable.

/max

>Hi all,
>
>On 05/18/2013 08:53 PM, Nick Boldt wrote:
>>base: test fail
>Base (which is used by almost all our component) has been red for more 
>than a week because of test failures or timeout. It has even been 
>branched for a revision that has been red for a week.
>What does that mean? It means that we will create a Beta1 based on a 
>component something that either has a bug, or that we don't know 
>whether it works or not. If I were Len or anyone else from the QE 
>team, I would be really annoyed by the fact that QE is about to be 
>shipped a build where base component is not guaranteed to work...
>
>This is a general bad habit we have, as a team. regarding automated 
>tests. Automated tests, on whichever component, are here to ensure the 
>quality of what we're doing, and to avoid respins or worse, bugs on 
>end-usage.
>These failed tests have to take priority #1 when they happen, and you 
>should monitor mails from CI or at the CI dashboard to check 
>everything you're working on seems to behave well. Leaving jobs into a 
>Red state is just taking a huge risk, which can have pretty bad 
>consequences (respins) and that will cost much more time and effort to 
>fix later.
>
>So please try to keep the CI jobs blue. It's our responsibility to all 
>of us to do good stuff, and to ensure we're doing good stuff.
>Cheers,
>-- 
>Mickael Istria
>Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools>
>My blog <http://mickaelistria.wordpress.com> - My Tweets 
><http://twitter.com/mickaelistria>

>_______________________________________________
>jbosstools-dev mailing list
>jbosstools-dev at lists.jboss.org
>https://lists.jboss.org/mailman/listinfo/jbosstools-dev