<div dir="ltr"><div><div><div><div><div>Hi,<br><br>to add few other observations I had with Jenkins vs TeamCity.<br><br></div>Biggest issue from my point of view is that Jenkins has a big performance issue when running more than one job same in concurrently on different nodes. <br>
</div>What happens is that when you have lets say 5 same jobs (with different parameters) running and first job would get to the end it would get stuck/wait in "recording test results" until all jobs are done and then all of them would finish in matter of minutes.<br>
<br></div><div>That causes big issue as build slaves don't do noting for in some cases hours as they are waiting for other jobs to complete. And new jobs are waiting for avalibale slaves, this result in hardware not being utilized as much as it could be.<br>
<br>Related recorded issues for this <br><a href="https://issues.jenkins-ci.org/browse/JENKINS-10234">https://issues.jenkins-ci.org/browse/JENKINS-10234</a><br><a href="https://issues.jenkins-ci.org/browse/JENKINS-10175">https://issues.jenkins-ci.org/browse/JENKINS-10175</a><br>
</div><br></div>Issues are open for few years now and the answer to it was "this is a feature not a bug" <br></div>everyone is recommending using xunit instead of junit plugin, but trick is that xunit plugin has totally different semantics and as such does not resolve our issue.<br>
<br><div><div><div>Other problem is general performance of building and amount it loads hardware. In general it takes 10-15% more time to do same job on jenkins then on TC when running on same hardware.<br><br></div><div>
This are just few deep technical issues I have found. <br></div><div><br>In any case i think everyone agrees that UI in general is much better with TC and also TC has nice integration with IDEs (eclipse, intellij, <a href="http://VS.NET">VS.NET</a>) which allow you to monitor/debug/start/review builds as they are executing.<br>
</div><div>That add a nice feature for developers to debug why their code is failing just one one build slave for example.<br><br><br>--<br></div><div>tomaz<br></div><div><div><div><br><br></div></div></div></div></div></div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jul 3, 2013 at 3:33 PM, Jaikiran Pai <span dir="ltr"><<a href="mailto:jpai@redhat.com" target="_blank">jpai@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div><tt>I'll let Tomaz answer that
question, but I'll add some points where I think TeamCity helped
a lot recently when compared to Jenkins. </tt><tt><br>
</tt>
<tt><br>
</tt>
<tt> Just before 8.0.0.Alpha2 of WildFly was released, we noticed
that our testsuite was in a very bad shape. Too many
intermittent failures. Although, the intermittent failures
weren't a new thing, the frequency and the number had both grown
to an extent where we felt that we had to start looking into
each of those tests and investigate the problems. I hadn't used
TeamCity earlier but while looking into these tests, I decided
to take a look at the instance maintained by Tomaz. From an user
point of view, I found the following features extremely handy
and in fact these features did help me with better investigating
the failures and also not losing interest in trying to track
down those test failures:</tt><tt><br>
</tt>
<tt><br>
</tt>
<tt> 1) "Investigations" feature - TeamCity has this feature
called "investigations" which allows you to mark a (failed) test
or an entire build for investigation. The investigation can be
assigned to a specific user. Investigations can be auto resolved
(the next time the build/test succeeds) or can be manually
resolved after investigating that failure. This feature allowed
me to keep track of a bunch of failing tests and monitor their
resolution over time. This is one step between finding a failing
test and creating a JIRA, since this intermediate step allowed
me to spend some time on that test to really understand what
needs to be fixed/changed for that test to pass. Once I knew
what was needed, I could then either fix it or file the JIRA
assigned to the relevant component/person.</tt><tt> This also
was one way of saying that this specific test failure is a
"known issue which is being investigated on by person X". This
way someone else can spend their time on some other test failure
investigation.</tt><tt><br>
</tt>
<tt><br>
</tt>
<tt>Investigations also allow "notes" to be attached to them which
allowed me to make a note of what I have investigated so far and
what might be the issue.</tt><tt><br>
</tt>
<tt><br>
</tt>
<tt>2) Immediate report and logs of failed tests - Unlike Jenkins
where you have to wait for the entire testsuite to finish (which
can take a hour and a half) to know how many and which tests
failed in that run, TeamCity shows the progress of the build and
reports the number of failed tests at that point in time in the
build. Furthermore, it shows logs and the failure details of
such tests immediately and you don't have to wait for the run to
complete. I found this extremely useful since I didn't have to
wait for the entire run to complete. In the past, when I've seen
intermittent failing tests on Jenkins, I haven't had the
determination to try out certain things and check the results
since the thought of having to wait for another hour and a half
would just switch off my interest on that issue.</tt><tt><br>
</tt>
<tt><br>
</tt>
<tt>3) Inline logs/stacktrace - I'm not sure why I like this so
much but I really do like this feature of TeamCity. This and #2
in themselves are the reasons which kept me interested in
tracking down a majority of the failures. This specific feature
is really simple. When a bunch of tests fail in a build, the
build report page shows all those failed test names and also for
each failed tests allows you to hide/show stacktrace inline
under the testcase name (see this for example
<a href="http://teamcity.cafe-babe.org/viewLog.html?buildId=5666&tab=buildResultsDiv&buildTypeId=pr" target="_blank">http://teamcity.cafe-babe.org/viewLog.html?buildId=5666&tab=buildResultsDiv&buildTypeId=pr</a>
- click on that test link and it will show up the logs inline
and you can then hide the logs if you want to). This allowed me
to view all those failed tests and their logs on the same page
and hide whichever ones I didn't want to view. Of course, in
Jenkins, you can view these logs on a separate page/tab for each
failed test, but I find the TeamCity way, much more usable than
the Jenkins way.</tt><tt><br>
</tt>
<tt><br>
</tt>
<tt>-Jaikiran</tt><div><div class="h5"><tt><br>
</tt>
<tt><br>
</tt><tt>On Tuesday 02 July 2013 11:50 PM, Vojtech Juranek wrote:</tt><tt><br>
</tt></div></div></div>
<blockquote type="cite"><div><div class="h5">
<pre>Hi,
</pre>
<blockquote type="cite">
<pre>it works very well.
In many cases much better than what we had with jenkins on lightning.
</pre>
</blockquote>
<pre>could you be more specific please? (not going to try to persuade you to stay
with Jenkins, just wondering what you see as Jenkins weak points and where is
TC better)
Thanks
Vojta</pre>
<tt><br>
</tt>
<fieldset></fieldset>
<tt><br>
</tt>
</div></div><pre>_______________________________________________
wildfly-dev mailing list
<a href="mailto:wildfly-dev@lists.jboss.org" target="_blank">wildfly-dev@lists.jboss.org</a>
<a href="https://lists.jboss.org/mailman/listinfo/wildfly-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
</pre>
</blockquote>
<tt><br>
</tt>
</div>
<br>_______________________________________________<br>
wildfly-dev mailing list<br>
<a href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/wildfly-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a><br>
<br></blockquote></div><br></div>