<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Feb 4, 2014 at 3:03 PM, Galder Zamarreņo <span dir="ltr"><<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="im"><br>
On 04 Feb 2014, at 13:50, Dan Berindei <<a href="mailto:dan.berindei@gmail.com">dan.berindei@gmail.com</a>> wrote:<br>
<br>
><br>
> On Tue, Feb 4, 2014 at 2:36 PM, Galder Zamarreņo <<a href="mailto:galder@redhat.com">galder@redhat.com</a>> wrote:<br>
> Narrowing down the list now, since this is a problem of how our CI is doing builds.<br>
><br>
> These logs are retrieved from [1].<br>
><br>
> Dunno how our CI is configured but this is odd. Seems like the build is halt due to test failures, but it continues somehow? I mean, the jars are not being produced properly, but the build is not halting.<br>
><br>
> We run the build with -fn (fail-never), so the build should never be halted because of a test failure. The configuration is here: <a href="http://ci.infinispan.org/admin/editRunType.html?id=buildType:bt8&runnerId=RUNNER_1" target="_blank">http://ci.infinispan.org/admin/editRunType.html?id=buildType:bt8&runnerId=RUNNER_1</a><br>
<br>
</div>^ That’s not working as expected, see the build log, my snippets…etc.<br></blockquote><div><br></div><div>Sorry, I didn't understand what's happening in those snippets. All I saw was an Ant script that doesn't do what it's supposed to do :)<br>
<br></div><div>I did see some differences in the configuration between the JDK6 and the JDK7 builds:<br>* JDK7 uses -fn and JDK6 uses -Dmaven.test.failure.ignore<br>* JDK7 uses -nsu (no snapshot updates), JDK6 doesn't<br>
<br></div><div>I've changed both builds to use -Dmaven.test.failure.ignore and -nsu, let's see how it goes.<br><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class="im"><br>
><br>
><br>
><br>
> It’s about time we did the following:<br>
> 1) Any test failures should halt the build there and then. IOW, do not continue the build at all.<br>
><br>
> Will having 100 tests in one run and 2000 tests in another really help?<br>
<br>
</div>As you disable randomly failing tests, and do not integrate commits making the testsuite fail, these number should even out.<br></blockquote><div><br></div><div>Not integrating commits that fail every time is easy, not integrating commits that fail randomly (maybe only in some environments) is trickier.<br>
<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class="im"><br>
><br>
><br>
> 2) Any tests that fail randomly should be disabled.<br>
><br>
> Let's go ahead and disable all the server tests then? ;)<br>
<br>
</div>Those server tests that are randomly failing should be disabled and looked at. Those tests that are failing as a result of container not starting are side effects of things not working properly, and these should not be disabled.<br>
</blockquote><div><br></div><div>Why treat the tests that are failing because of a build problem differently? What about the tests that fail only on IBM JDK6?<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class=""><div class="h5"><br>
><br>
><br>
><br>
> Cheers,<br>
><br>
> [1] <a href="https://dl.dropboxusercontent.com/u/6148072/does-not-work.log" target="_blank">https://dl.dropboxusercontent.com/u/6148072/does-not-work.log</a><br>
><br>
> On 04 Feb 2014, at 13:30, Galder Zamarreņo <<a href="mailto:galder@redhat.com">galder@redhat.com</a>> wrote:<br>
><br>
> ><br>
> > On 04 Feb 2014, at 10:38, Stuart Douglas <<a href="mailto:stuart.w.douglas@gmail.com">stuart.w.douglas@gmail.com</a>> wrote:<br>
> ><br>
> >> It is almost certainly something to do with this:<br>
> >><br>
> >> <module-def name="org.infinispan.server.rest" src="${infinispan.server.modules.dir}"><br>
> >><br>
> >> <maven-resource-with-classifier group="org.infinispan" artifact="infinispan-server-rest" classifier="classes" /><br>
> >><br>
> >> </module-def><br>
> >><br>
> >> I guess sometimes the classes artefact is being attached as a reference to the classes directory, rather than a reference to a jar, which causes the issue.<br>
> ><br>
> > Here’s a gist with a subset of the build log [1]. When it works fine, it’s copying a jar, when it’s not, it’s copying an empty folder.<br>
> ><br>
> > However, this is not only happening for the org.infinispan.server.rest module, others show the same issue [2]. What seems to be a pattern is that it only happens with modules that are built by us, it’s not happening for modules coming with the base AS/WF instance.<br>
> ><br>
> > I’ve traced back and this might be due to build failures that are not producing the right jars [3].<br>
> ><br>
> > @Stuart, this is really our problem. Sorry for the inconvenience!<br>
> ><br>
> > [1] <a href="https://gist.github.com/galderz/b9286f385aad1316df51" target="_blank">https://gist.github.com/galderz/b9286f385aad1316df51</a><br>
> > [2] <a href="https://gist.github.com/galderz/9e6a9bd9b18b805db323" target="_blank">https://gist.github.com/galderz/9e6a9bd9b18b805db323</a><br>
> > [3] <a href="https://gist.github.com/galderz/6ab662a1027cd96cbd8c" target="_blank">https://gist.github.com/galderz/6ab662a1027cd96cbd8c</a><br>
> ><br>
> >><br>
> >> Stuart<br>
> >><br>
> >><br>
> >> On Tue, Feb 4, 2014 at 11:14 AM, Galder Zamarreņo <<a href="mailto:galder@redhat.com">galder@redhat.com</a>> wrote:<br>
> >><br>
> >> On 04 Feb 2014, at 10:01, Stuart Douglas <<a href="mailto:stuart.w.douglas@gmail.com">stuart.w.douglas@gmail.com</a>> wrote:<br>
> >><br>
> >>><br>
> >>><br>
> >>><br>
> >>> On Tue, Feb 4, 2014 at 11:00 AM, Stuart Douglas <<a href="mailto:stuart.w.douglas@gmail.com">stuart.w.douglas@gmail.com</a>> wrote:<br>
> >>> Yes, there is nothing in the server code that modified the modules directory.<br>
> >>><br>
> >>> Well, except for the new patching stuff, but that is not really relevant here.<br>
> >><br>
> >> The testsuite AS/WF builds are built out of the distribution build, which shows the same problem. The distribution we build uses the scripts we got from AS [1].<br>
> >><br>
> >> Do you see anything in there that could be causing this? We are using maven-antrun-plugin version 1.3, and take into account the lib.xml in [2].<br>
> >><br>
> >> Finally, do you have any suggestions on changes we could make to these files to further debug the issue?<br>
> >><br>
> >> Thanks a lot for your help!<br>
> >><br>
> >> [1] <a href="https://github.com/infinispan/infinispan/blob/master/server/integration/build/build.xml" target="_blank">https://github.com/infinispan/infinispan/blob/master/server/integration/build/build.xml</a><br>
> >> [2] <a href="https://github.com/infinispan/infinispan/blob/master/server/integration/build/lib.xml" target="_blank">https://github.com/infinispan/infinispan/blob/master/server/integration/build/lib.xml</a><br>
> >><br>
> >>><br>
> >>> Stuart<br>
> >>><br>
> >>><br>
> >>> Stuart<br>
> >>><br>
> >>><br>
> >>> On Tue, Feb 4, 2014 at 10:56 AM, Galder Zamarreņo <<a href="mailto:galder@redhat.com">galder@redhat.com</a>> wrote:<br>
> >>><br>
> >>> On 04 Feb 2014, at 09:37, Stuart Douglas <<a href="mailto:stuart.w.douglas@gmail.com">stuart.w.douglas@gmail.com</a>> wrote:<br>
> >>><br>
> >>>> This looks like an issue with your environment. The modules directory is static. Wildfly does not contain any code that messes with it. I would say the culprit is probably something in either your build process or your test suite.<br>
> >>><br>
> >>> Correction, this is happening with AS 7.2.0.Final (Wildfly 8 used somewhere else). I guess your answer still applies?<br>
> >>><br>
> >>> Cheers,<br>
> >>><br>
> >>>><br>
> >>>> Stuart<br>
> >>>><br>
> >>>><br>
> >>>> On Tue, Feb 4, 2014 at 10:21 AM, Galder Zamarreņo <<a href="mailto:galder@redhat.com">galder@redhat.com</a>> wrote:<br>
> >>>> Hi all,<br>
> >>>><br>
> >>>> We’re having issues with our Infinispan Server integration tests, which run within Wildfly 8.0.0.Beta1 (as I’m typing I’m wondering if we should just upgrade it to see if this goes away...?).<br>
> >>>><br>
> >>>> Quite often some of the runs fail with error message [1].<br>
> >>>><br>
> >>>> Having looked at the build environment when a run fails, you see this:<br>
> >>>><br>
> >>>> --<br>
> >>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main<br>
> >>>> drwxrwxr-x 2 g staff 68B Feb 3 18:41 classes (<— a directory??)<br>
> >>>> -rw-r--r-- 1 g staff 1B Feb 3 18:41 classes.index<br>
> >>>> -rw-r--r-- 1 g staff 2.1K Feb 3 18:41 module.xml<br>
> >>>><br>
> >>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main/classes<br>
> >>>> drwxrwxr-x 2 g staff 68B Feb 3 18:41 .<br>
> >>>> drwxrwxr-x 5 g staff 170B Feb 3 18:41 ..<br>
> >>>><br>
> >>>> $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml<br>
> >>>> <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”><br>
> >>>> ...<br>
> >>>> <resource-root path="classes"/><br>
> >>>> ...<br>
> >>>><br>
> >>>> This is completely different to what happens with a successful run:<br>
> >>>><br>
> >>>> --<br>
> >>>> $ ls modules/system/layers/base/org/infinispan/server/rest/main<br>
> >>>> -rw-r--r-- 1 g staff 103K Feb 3 19:40 infinispan-classes.jar (<— a jar file!)<br>
> >>>> -rw-r--r-- 1 g staff 278B Feb 3 19:40 infinispan-classes.jar.index<br>
> >>>> -rw-r--r-- 1 g staff 2.1K Feb 3 19:40 module.xml<br>
> >>>><br>
> >>>> $ jar tf modules/system/layers/base/org/infinispan/server/rest/main/infinispan-classes.jar | grep ExtendedHeaders<br>
> >>>> org/infinispan/rest/configuration/ExtendedHeaders.class<br>
> >>>><br>
> >>>> $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml<br>
> >>>> <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”><br>
> >>>> ...<br>
> >>>> <resource-root path="infinispan-classes.jar”/><br>
> >>>> —<br>
> >>>><br>
> >>>> Anyone can explain what is going on here? Does it ring a bell to anyone? Is this a known Wildfly issue by any chance?<br>
> >>>><br>
> >>>> [1] <a href="https://gist.github.com/galderz/bd74cebfc840ef3ae284" target="_blank">https://gist.github.com/galderz/bd74cebfc840ef3ae284</a><br>
> >>>><br>
> >>>> Cheers,<br>
> >>>> --<br>
> >>>> Galder Zamarreņo<br>
> >>>> <a href="mailto:galder@redhat.com">galder@redhat.com</a><br>
> >>>> <a href="http://twitter.com/galderz" target="_blank">twitter.com/galderz</a><br>
> >>>><br>
> >>>> Project Lead, Escalante<br>
> >>>> <a href="http://escalante.io" target="_blank">http://escalante.io</a><br>
> >>>><br>
> >>>> Engineer, Infinispan<br>
> >>>> <a href="http://infinispan.org" target="_blank">http://infinispan.org</a><br>
> >>>><br>
> >>>><br>
> >>>> _______________________________________________<br>
> >>>> jboss-as7-dev mailing list<br>
> >>>> <a href="mailto:jboss-as7-dev@lists.jboss.org">jboss-as7-dev@lists.jboss.org</a><br>
> >>>> <a href="https://lists.jboss.org/mailman/listinfo/jboss-as7-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/jboss-as7-dev</a><br>
> >>>><br>
> >>><br>
> >>><br>
> >>> --<br>
> >>> Galder Zamarreņo<br>
> >>> <a href="mailto:galder@redhat.com">galder@redhat.com</a><br>
> >>> <a href="http://twitter.com/galderz" target="_blank">twitter.com/galderz</a><br>
> >>><br>
> >>> Project Lead, Escalante<br>
> >>> <a href="http://escalante.io" target="_blank">http://escalante.io</a><br>
> >>><br>
> >>> Engineer, Infinispan<br>
> >>> <a href="http://infinispan.org" target="_blank">http://infinispan.org</a><br>
> >>><br>
> >>><br>
> >>><br>
> >><br>
> >><br>
> >> --<br>
> >> Galder Zamarreņo<br>
> >> <a href="mailto:galder@redhat.com">galder@redhat.com</a><br>
> >> <a href="http://twitter.com/galderz" target="_blank">twitter.com/galderz</a><br>
> >><br>
> >> Project Lead, Escalante<br>
> >> <a href="http://escalante.io" target="_blank">http://escalante.io</a><br>
> >><br>
> >> Engineer, Infinispan<br>
> >> <a href="http://infinispan.org" target="_blank">http://infinispan.org</a><br>
> >><br>
> >><br>
> ><br>
> ><br>
> > --<br>
> > Galder Zamarreņo<br>
> > <a href="mailto:galder@redhat.com">galder@redhat.com</a><br>
> > <a href="http://twitter.com/galderz" target="_blank">twitter.com/galderz</a><br>
> ><br>
> > Project Lead, Escalante<br>
> > <a href="http://escalante.io" target="_blank">http://escalante.io</a><br>
> ><br>
> > Engineer, Infinispan<br>
> > <a href="http://infinispan.org" target="_blank">http://infinispan.org</a><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > infinispan-dev mailing list<br>
> > <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
> > <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
><br>
><br>
> --<br>
> Galder Zamarreņo<br>
> <a href="mailto:galder@redhat.com">galder@redhat.com</a><br>
> <a href="http://twitter.com/galderz" target="_blank">twitter.com/galderz</a><br>
><br>
> Project Lead, Escalante<br>
> <a href="http://escalante.io" target="_blank">http://escalante.io</a><br>
><br>
> Engineer, Infinispan<br>
> <a href="http://infinispan.org" target="_blank">http://infinispan.org</a><br>
><br>
><br>
> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
><br>
> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
<br>
<br>
--<br>
Galder Zamarreņo<br>
<a href="mailto:galder@redhat.com">galder@redhat.com</a><br>
<a href="http://twitter.com/galderz" target="_blank">twitter.com/galderz</a><br>
<br>
Project Lead, Escalante<br>
<a href="http://escalante.io" target="_blank">http://escalante.io</a><br>
<br>
Engineer, Infinispan<br>
<a href="http://infinispan.org" target="_blank">http://infinispan.org</a><br>
<br>
<br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</div></div></blockquote></div><br></div></div>