[wildfly-dev] Pull request building improvements

Mon Jul 8 10:23:41 EDT 2013

Thanks for the feedback. While taking responsibility for failed test is IMHO 
addressed by Claim plugin [1], the rest are useful proposals for better 
Jenkins UX, now logged as [2,3,4]. If someone here is still  interested in 
Jenkins, feel free to log other proposals under JBQA-8165 [5].
Thanks
Vojta

[1] https://wiki.jenkins-ci.org/display/JENKINS/Claim+plugin
[2] https://issues.jboss.org/browse/JBQA-8167
[3] https://issues.jboss.org/browse/JBQA-8168
[4] https://issues.jboss.org/browse/JBQA-8169
[5] https://issues.jboss.org/browse/JBQA-8165

On Wednesday 03 July 2013 09:43:11 Jason Greene wrote:
> One of the other things was that Jenkins uses stats that are relative to the
> last good build, which makes them useless for determining how often an
> intermittent test fails. With TC we immediately know what the top abusers
> are.
> On Jul 3, 2013, at 8:33 AM, Jaikiran Pai <jpai at redhat.com> wrote:
> > I'll let Tomaz answer that question, but I'll add some points where I
> > think TeamCity helped a lot recently when compared to Jenkins.
> > 
> > Just before 8.0.0.Alpha2 of WildFly was released, we noticed that our
> > testsuite was in a very bad shape. Too many intermittent failures.
> > Although, the intermittent failures weren't a new thing, the frequency
> > and the number had both grown to an extent where we felt that we had to
> > start looking into each of those tests and investigate the problems. I
> > hadn't used TeamCity earlier but while looking into these tests, I
> > decided to take a look at the instance maintained by Tomaz. From an user
> > point of view, I found the following features extremely handy and in fact
> > these features did help me with better investigating the failures and
> > also not losing interest in trying to track down those test failures:
> > 
> > 1) "Investigations" feature -  TeamCity has this feature called
> > "investigations" which allows you to mark a (failed) test or an entire
> > build for investigation. The investigation can be assigned to a specific
> > user. Investigations can be auto resolved (the next time the build/test
> > succeeds) or can be manually resolved after investigating that failure.
> > This feature allowed me to keep track of a bunch of failing tests and
> > monitor their resolution over time. This is one step between finding a
> > failing test and creating a JIRA, since this intermediate step allowed me
> > to spend some time on that test to really understand what needs to be
> > fixed/changed for that test to pass. Once I knew what was needed, I could
> > then either fix it or file the JIRA assigned to the relevant
> > component/person. This also was one way of saying that this specific test
> > failure is a "known issue which is being investigated on by person X".
> > This way someone else can spend their time on some other test failure
> > investigation.
> > 
> > Investigations also allow "notes" to be attached to them which allowed me
> > to make a note of what I have investigated so far and what might be the
> > issue.
> > 
> > 2) Immediate report and logs of failed tests - Unlike Jenkins where you
> > have to wait for the entire testsuite to finish (which can take a hour
> > and a half) to know how many and which tests failed in that run, TeamCity
> > shows the progress of the build and reports the number of failed tests at
> > that point in time in the build. Furthermore, it shows logs and the
> > failure details of such tests immediately and you don't have to wait for
> > the run to complete. I found this extremely useful since I didn't have to
> > wait for the entire run to complete. In the past, when I've seen
> > intermittent failing tests on Jenkins, I haven't had the determination to
> > try out certain things and check the results since the thought of having
> > to wait for another hour and a half would just switch off my interest on
> > that issue.
> > 
> > 3) Inline logs/stacktrace - I'm not sure why I like this so much but I
> > really do like this feature of TeamCity. This and #2 in themselves are
> > the reasons which kept me interested in tracking down a majority of the
> > failures. This specific feature is really simple. When a bunch of tests
> > fail in a build, the build report page shows all those failed test names
> > and also for each failed tests allows you to hide/show stacktrace inline
> > under the testcase name (see this for example
> > http://teamcity.cafe-babe.org/viewLog.html?buildId=5666&tab=buildResultsD
> > iv&buildTypeId=pr - click on that test link and it will show up the logs
> > inline and you can then hide the logs if you want to). This allowed me to
> > view all those failed tests and their logs on the same page and hide
> > whichever ones I didn't want to view. Of course, in Jenkins, you can view
> > these logs on a separate page/tab for each failed test, but I find the
> > TeamCity way, much more usable than the Jenkins way.
> > 
> > -Jaikiran
> > 
> > On Tuesday 02 July 2013 11:50 PM, Vojtech Juranek wrote:
> >> Hi,
> >> 
> >>> it works very well.
> >>> In many cases much better than what we had with jenkins on lightning.
> >> 
> >> could you be more specific please? (not going to try to persuade you to
> >> stay with Jenkins, just wondering what you see as Jenkins weak points
> >> and where is TC better)
> >> 
> >> Thanks
> >> Vojta
> >> 
> >> 
> >> 
> >> 
> >> _______________________________________________
> >> wildfly-dev mailing list
> >> 
> >> wildfly-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> > 
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect
> JBoss, a division of Red Hat
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part.
Url : http://lists.jboss.org/pipermail/wildfly-dev/attachments/20130708/840a7f59/attachment.bin 