<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix"><tt>Thanks Vojta for filing those
JIRAs. I just had a quick look at the Claim plugin overview and
it doesn't look like it allows the investigation/claim to be
done on a per testcase basis. Does it have that support too?</tt><tt><br>
</tt><tt><br>
</tt><tt>-Jaikiran</tt><tt><br>
</tt><tt>On Monday 08 July 2013 07:53 PM, Vojtech Juranek wrote:</tt><tt><br>
</tt></div>
<blockquote cite="mid:3774771.3YPfjn2rie@dhcp-4-219.brq.redhat.com"
type="cite">
<pre wrap="">Thanks for the feedback. While taking responsibility for failed test is IMHO
addressed by Claim plugin [1], the rest are useful proposals for better
Jenkins UX, now logged as [2,3,4]. If someone here is still interested in
Jenkins, feel free to log other proposals under JBQA-8165 [5].
Thanks
Vojta
[1] <a class="moz-txt-link-freetext" href="https://wiki.jenkins-ci.org/display/JENKINS/Claim+plugin">https://wiki.jenkins-ci.org/display/JENKINS/Claim+plugin</a>
[2] <a class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/JBQA-8167">https://issues.jboss.org/browse/JBQA-8167</a>
[3] <a class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/JBQA-8168">https://issues.jboss.org/browse/JBQA-8168</a>
[4] <a class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/JBQA-8169">https://issues.jboss.org/browse/JBQA-8169</a>
[5] <a class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/JBQA-8165">https://issues.jboss.org/browse/JBQA-8165</a>
On Wednesday 03 July 2013 09:43:11 Jason Greene wrote:
</pre>
<blockquote type="cite">
<pre wrap="">One of the other things was that Jenkins uses stats that are relative to the
last good build, which makes them useless for determining how often an
intermittent test fails. With TC we immediately know what the top abusers
are.
On Jul 3, 2013, at 8:33 AM, Jaikiran Pai <a class="moz-txt-link-rfc2396E" href="mailto:jpai@redhat.com"><jpai@redhat.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I'll let Tomaz answer that question, but I'll add some points where I
think TeamCity helped a lot recently when compared to Jenkins.
Just before 8.0.0.Alpha2 of WildFly was released, we noticed that our
testsuite was in a very bad shape. Too many intermittent failures.
Although, the intermittent failures weren't a new thing, the frequency
and the number had both grown to an extent where we felt that we had to
start looking into each of those tests and investigate the problems. I
hadn't used TeamCity earlier but while looking into these tests, I
decided to take a look at the instance maintained by Tomaz. From an user
point of view, I found the following features extremely handy and in fact
these features did help me with better investigating the failures and
also not losing interest in trying to track down those test failures:
1) "Investigations" feature - TeamCity has this feature called
"investigations" which allows you to mark a (failed) test or an entire
build for investigation. The investigation can be assigned to a specific
user. Investigations can be auto resolved (the next time the build/test
succeeds) or can be manually resolved after investigating that failure.
This feature allowed me to keep track of a bunch of failing tests and
monitor their resolution over time. This is one step between finding a
failing test and creating a JIRA, since this intermediate step allowed me
to spend some time on that test to really understand what needs to be
fixed/changed for that test to pass. Once I knew what was needed, I could
then either fix it or file the JIRA assigned to the relevant
component/person. This also was one way of saying that this specific test
failure is a "known issue which is being investigated on by person X".
This way someone else can spend their time on some other test failure
investigation.
Investigations also allow "notes" to be attached to them which allowed me
to make a note of what I have investigated so far and what might be the
issue.
2) Immediate report and logs of failed tests - Unlike Jenkins where you
have to wait for the entire testsuite to finish (which can take a hour
and a half) to know how many and which tests failed in that run, TeamCity
shows the progress of the build and reports the number of failed tests at
that point in time in the build. Furthermore, it shows logs and the
failure details of such tests immediately and you don't have to wait for
the run to complete. I found this extremely useful since I didn't have to
wait for the entire run to complete. In the past, when I've seen
intermittent failing tests on Jenkins, I haven't had the determination to
try out certain things and check the results since the thought of having
to wait for another hour and a half would just switch off my interest on
that issue.
3) Inline logs/stacktrace - I'm not sure why I like this so much but I
really do like this feature of TeamCity. This and #2 in themselves are
the reasons which kept me interested in tracking down a majority of the
failures. This specific feature is really simple. When a bunch of tests
fail in a build, the build report page shows all those failed test names
and also for each failed tests allows you to hide/show stacktrace inline
under the testcase name (see this for example
<a class="moz-txt-link-freetext" href="http://teamcity.cafe-babe.org/viewLog.html?buildId=5666&tab=buildResultsD">http://teamcity.cafe-babe.org/viewLog.html?buildId=5666&tab=buildResultsD</a>
iv&buildTypeId=pr - click on that test link and it will show up the logs
inline and you can then hide the logs if you want to). This allowed me to
view all those failed tests and their logs on the same page and hide
whichever ones I didn't want to view. Of course, in Jenkins, you can view
these logs on a separate page/tab for each failed test, but I find the
TeamCity way, much more usable than the Jenkins way.
-Jaikiran
On Tuesday 02 July 2013 11:50 PM, Vojtech Juranek wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
</pre>
<blockquote type="cite">
<pre wrap="">it works very well.
In many cases much better than what we had with jenkins on lightning.
</pre>
</blockquote>
<pre wrap="">
could you be more specific please? (not going to try to persuade you to
stay with Jenkins, just wondering what you see as Jenkins weak points
and where is TC better)
Thanks
Vojta
_______________________________________________
wildfly-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/wildfly-dev">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
</pre>
</blockquote>
<pre wrap="">
_______________________________________________
wildfly-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/wildfly-dev">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
</pre>
</blockquote>
<pre wrap="">
--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat</pre>
<tt><br>
</tt>
<fieldset class="mimeAttachmentHeader"></fieldset>
<tt><br>
</tt>
<pre wrap="">_______________________________________________
wildfly-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/wildfly-dev">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
</pre>
</blockquote>
</blockquote>
<tt><br>
</tt>
</body>
</html>