[wildfly-dev] Pull request building improvements

Mon Jul 8 10:33:38 EDT 2013

Thanks Vojta for filing those JIRAs. I just had a quick look at the 
Claim plugin overview and it doesn't look like it allows the 
investigation/claim to be done on a per testcase basis. Does it have 
that support too?

-Jaikiran
On Monday 08 July 2013 07:53 PM, Vojtech Juranek wrote:
> Thanks for the feedback. While taking responsibility for failed test is IMHO
> addressed by Claim plugin [1], the rest are useful proposals for better
> Jenkins UX, now logged as [2,3,4]. If someone here is still  interested in
> Jenkins, feel free to log other proposals under JBQA-8165 [5].
> Thanks
> Vojta
>
> [1] https://wiki.jenkins-ci.org/display/JENKINS/Claim+plugin
> [2] https://issues.jboss.org/browse/JBQA-8167
> [3] https://issues.jboss.org/browse/JBQA-8168
> [4] https://issues.jboss.org/browse/JBQA-8169
> [5] https://issues.jboss.org/browse/JBQA-8165
>
> On Wednesday 03 July 2013 09:43:11 Jason Greene wrote:
>> One of the other things was that Jenkins uses stats that are relative to the
>> last good build, which makes them useless for determining how often an
>> intermittent test fails. With TC we immediately know what the top abusers
>> are.
>> On Jul 3, 2013, at 8:33 AM, Jaikiran Pai <jpai at redhat.com> wrote:
>>> I'll let Tomaz answer that question, but I'll add some points where I
>>> think TeamCity helped a lot recently when compared to Jenkins.
>>>
>>> Just before 8.0.0.Alpha2 of WildFly was released, we noticed that our
>>> testsuite was in a very bad shape. Too many intermittent failures.
>>> Although, the intermittent failures weren't a new thing, the frequency
>>> and the number had both grown to an extent where we felt that we had to
>>> start looking into each of those tests and investigate the problems. I
>>> hadn't used TeamCity earlier but while looking into these tests, I
>>> decided to take a look at the instance maintained by Tomaz. From an user
>>> point of view, I found the following features extremely handy and in fact
>>> these features did help me with better investigating the failures and
>>> also not losing interest in trying to track down those test failures:
>>>
>>> 1) "Investigations" feature -  TeamCity has this feature called
>>> "investigations" which allows you to mark a (failed) test or an entire
>>> build for investigation. The investigation can be assigned to a specific
>>> user. Investigations can be auto resolved (the next time the build/test
>>> succeeds) or can be manually resolved after investigating that failure.
>>> This feature allowed me to keep track of a bunch of failing tests and
>>> monitor their resolution over time. This is one step between finding a
>>> failing test and creating a JIRA, since this intermediate step allowed me
>>> to spend some time on that test to really understand what needs to be
>>> fixed/changed for that test to pass. Once I knew what was needed, I could
>>> then either fix it or file the JIRA assigned to the relevant
>>> component/person. This also was one way of saying that this specific test
>>> failure is a "known issue which is being investigated on by person X".
>>> This way someone else can spend their time on some other test failure
>>> investigation.
>>>
>>> Investigations also allow "notes" to be attached to them which allowed me
>>> to make a note of what I have investigated so far and what might be the
>>> issue.
>>>
>>> 2) Immediate report and logs of failed tests - Unlike Jenkins where you
>>> have to wait for the entire testsuite to finish (which can take a hour
>>> and a half) to know how many and which tests failed in that run, TeamCity
>>> shows the progress of the build and reports the number of failed tests at
>>> that point in time in the build. Furthermore, it shows logs and the
>>> failure details of such tests immediately and you don't have to wait for
>>> the run to complete. I found this extremely useful since I didn't have to
>>> wait for the entire run to complete. In the past, when I've seen
>>> intermittent failing tests on Jenkins, I haven't had the determination to
>>> try out certain things and check the results since the thought of having
>>> to wait for another hour and a half would just switch off my interest on
>>> that issue.
>>>
>>> 3) Inline logs/stacktrace - I'm not sure why I like this so much but I
>>> really do like this feature of TeamCity. This and #2 in themselves are
>>> the reasons which kept me interested in tracking down a majority of the
>>> failures. This specific feature is really simple. When a bunch of tests
>>> fail in a build, the build report page shows all those failed test names
>>> and also for each failed tests allows you to hide/show stacktrace inline
>>> under the testcase name (see this for example
>>> http://teamcity.cafe-babe.org/viewLog.html?buildId=5666&tab=buildResultsD
>>> iv&buildTypeId=pr - click on that test link and it will show up the logs
>>> inline and you can then hide the logs if you want to). This allowed me to
>>> view all those failed tests and their logs on the same page and hide
>>> whichever ones I didn't want to view. Of course, in Jenkins, you can view
>>> these logs on a separate page/tab for each failed test, but I find the
>>> TeamCity way, much more usable than the Jenkins way.
>>>
>>> -Jaikiran
>>>
>>> On Tuesday 02 July 2013 11:50 PM, Vojtech Juranek wrote:
>>>> Hi,
>>>>
>>>>> it works very well.
>>>>> In many cases much better than what we had with jenkins on lightning.
>>>> could you be more specific please? (not going to try to persuade you to
>>>> stay with Jenkins, just wondering what you see as Jenkins weak points
>>>> and where is TC better)
>>>>
>>>> Thanks
>>>> Vojta
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>>
>>>> wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> --
>> Jason T. Greene
>> WildFly Lead / JBoss EAP Platform Architect
>> JBoss, a division of Red Hat
>>
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20130708/95525f09/attachment.html 