[jbosstools-dev] Test failures/timeout are very bad

[Victor Rubezhny]

+1 for this change

Regards,
Victor

On 05/23/2013 12:42 AM, Snjezana Peco wrote:
> We would need to apply
> https://github.com/jbosstools/jbosstools-build/pull/88. See
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=405456
> The test failure at
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JBossTools/view/JBossTools_Trunk/job/jbosstools-base_master/260/testReport/org.jboss.tools.common.model.test/ClassPathTest/testSharingJarContent/
> is caused by this bug.
>
> Snjeza
>
> On 5/21/2013 2:42 PM, Max Rydahl Andersen wrote:
>> On Tue, May 21, 2013 at 10:46:24AM +0200, Mickael Istria wrote:
>>
>> Mickael - problem is that these tests passes fine on a local machine.
>>
>> Fails on jenkins which unfortunately keeps being extremly unreliable and slow.
>>
>> But Yes - we need to fix these and adjust the tests to not be reliant
>> on multiple jobs/threads etc. so the tests can run to completeness more reliable.
>>
>> thats the worst thing I see in our testsuite, the tests are relying on concurrent jobs
>> and aren't simply testing a unit of code. it's testing a combination (i.e. big integration test)
>> and that's not something we should have in our core tests - thats too unreliable.
>>
>> /max
>>
>>> Hi all,
>>>
>>> On 05/18/2013 08:53 PM, Nick Boldt wrote:
>>>> base: test fail
>>> Base (which is used by almost all our component) has been red for more
>>> than a week because of test failures or timeout. It has even been
>>> branched for a revision that has been red for a week.
>>> What does that mean? It means that we will create a Beta1 based on a
>>> component something that either has a bug, or that we don't know
>>> whether it works or not. If I were Len or anyone else from the QE
>>> team, I would be really annoyed by the fact that QE is about to be
>>> shipped a build where base component is not guaranteed to work...
>>>
>>> This is a general bad habit we have, as a team. regarding automated
>>> tests. Automated tests, on whichever component, are here to ensure the
>>> quality of what we're doing, and to avoid respins or worse, bugs on
>>> end-usage.
>>> These failed tests have to take priority #1 when they happen, and you
>>> should monitor mails from CI or at the CI dashboard to check
>>> everything you're working on seems to behave well. Leaving jobs into a
>>> Red state is just taking a huge risk, which can have pretty bad
>>> consequences (respins) and that will cost much more time and effort to
>>> fix later.
>>>
>>> So please try to keep the CI jobs blue. It's our responsibility to all
>>> of us to do good stuff, and to ensure we're doing good stuff.
>>> Cheers,
>>> -- 
>>> Mickael Istria
>>> Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools>
>>> My blog <http://mickaelistria.wordpress.com> - My Tweets
>>> <http://twitter.com/mickaelistria>
>>> _______________________________________________
>>> jbosstools-dev mailing list
>>> jbosstools-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>> _______________________________________________
>> jbosstools-dev mailing list
>> jbosstools-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
> _______________________________________________
> jbosstools-dev mailing list
> jbosstools-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosstools-dev