Re: [jbosstools-dev] Test failures/timeout are very bad

On Tue, May 21, 2013 at 10:46:24AM +0200, Mickael Istria wrote: Mickael - problem is that these tests passes fine on a local machine. Fails on jenkins which unfortunately keeps being extremly unreliable and slow. But Yes - we need to fix these and adjust the tests to not be reliant on multiple jobs/threads etc. so the tests can run to completeness more reliable. thats the worst thing I see in our testsuite, the tests are relying on concurrent jobs and aren't simply testing a unit of code. it's testing a combination (i.e. big integration test) and that's not something we should have in our core tests - thats too unreliable. /max > Hi all, > > On 05/18/2013 08:53 PM, Nick Boldt wrote: >> base: test fail > Base (which is used by almost all our component) has been red for more > than a week because of test failures or timeout. It has even been > branched for a revision that has been red for a week. > What does that mean? It means that we will create a Beta1 based on a > component something that either has a bug, or that we don't know > whether it works or not. If I were Len or anyone else from the QE > team, I would be really annoyed by the fact that QE is about to be > shipped a build where base component is not guaranteed to work... > > This is a general bad habit we have, as a team. regarding automated > tests. Automated tests, on whichever component, are here to ensure the > quality of what we're doing, and to avoid respins or worse, bugs on > end-usage. > These failed tests have to take priority #1 when they happen, and you > should monitor mails from CI or at the CI dashboard to check > everything you're working on seems to behave well. Leaving jobs into a > Red state is just taking a huge risk, which can have pretty bad > consequences (respins) and that will cost much more time and effort to > fix later. > > So please try to keep the CI jobs blue. It's our responsibility to all > of us to do good stuff, and to ensure we're doing good stuff. > Cheers, > -- > Mickael Istria > Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools> > My blog <http://mickaelistria.wordpress.com> - My Tweets > <http://twitter.com/mickaelistria> > _______________________________________________ > jbosstools-dev mailing list > jbosstools-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jbosstools-dev _______________________________________________ jbosstools-dev mailing list jbosstools-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosstools-dev