[jbosstools-dev] Test failures/timeout are very bad

[Denis Golovin]

On 05/22/2013 01:42 PM, Snjezana Peco wrote:
> We would need to apply
> https://github.com/jbosstools/jbosstools-build/pull/88. See
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=405456
> The test failure at
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JBossTools/view/JBossTools_Trunk/job/jbosstools-base_master/260/testReport/org.jboss.tools.common.model.test/ClassPathTest/testSharingJarContent/
> is caused by this bug.
The real cause of this issue is xvfb started to provide display for 
tests, switching to xvnc solves the issue. The same problem was 
discovered for hibernate tests, even disabling workspace auto save job 
through plugin_customization.ini didn't solve the issue and build stuck 
anyway in Display.sleep() method forever.

Denis
>
> Snjeza
>
> On 5/21/2013 2:42 PM, Max Rydahl Andersen wrote:
>> On Tue, May 21, 2013 at 10:46:24AM +0200, Mickael Istria wrote:
>>
>> Mickael - problem is that these tests passes fine on a local machine.
>>
>> Fails on jenkins which unfortunately keeps being extremly unreliable and slow.
>>
>> But Yes - we need to fix these and adjust the tests to not be reliant
>> on multiple jobs/threads etc. so the tests can run to completeness more reliable.
>>
>> thats the worst thing I see in our testsuite, the tests are relying on concurrent jobs
>> and aren't simply testing a unit of code. it's testing a combination (i.e. big integration test)
>> and that's not something we should have in our core tests - thats too unreliable.
>>
>> /max
>>
>>> Hi all,
>>>
>>> On 05/18/2013 08:53 PM, Nick Boldt wrote:
>>>> base: test fail
>>> Base (which is used by almost all our component) has been red for more
>>> than a week because of test failures or timeout. It has even been
>>> branched for a revision that has been red for a week.
>>> What does that mean? It means that we will create a Beta1 based on a
>>> component something that either has a bug, or that we don't know
>>> whether it works or not. If I were Len or anyone else from the QE
>>> team, I would be really annoyed by the fact that QE is about to be
>>> shipped a build where base component is not guaranteed to work...
>>>
>>> This is a general bad habit we have, as a team. regarding automated
>>> tests. Automated tests, on whichever component, are here to ensure the
>>> quality of what we're doing, and to avoid respins or worse, bugs on
>>> end-usage.
>>> These failed tests have to take priority #1 when they happen, and you
>>> should monitor mails from CI or at the CI dashboard to check
>>> everything you're working on seems to behave well. Leaving jobs into a
>>> Red state is just taking a huge risk, which can have pretty bad
>>> consequences (respins) and that will cost much more time and effort to
>>> fix later.
>>>
>>> So please try to keep the CI jobs blue. It's our responsibility to all
>>> of us to do good stuff, and to ensure we're doing good stuff.
>>> Cheers,
>>> -- 
>>> Mickael Istria
>>> Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools>
>>> My blog <http://mickaelistria.wordpress.com> - My Tweets
>>> <http://twitter.com/mickaelistria>
>>> _______________________________________________
>>> jbosstools-dev mailing list
>>> jbosstools-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>> _______________________________________________
>> jbosstools-dev mailing list
>> jbosstools-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
> _______________________________________________
> jbosstools-dev mailing list
> jbosstools-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosstools-dev