[infinispan-dev] Hidden failures in the testsuite

[Sanne Grinovero]

Hi all,
sorry for the slow reply, I can't get to test Infinispan very often lately.

On top of previously reported issues - which I still have - today I
also noticed this one:

[UnitTestTestNGListener] Test
testPutTimeout(org.infinispan.client.hotrod.ClientSocketReadTimeoutTest)
failed.
Sep 09, 2015 11:28:16 AM
io.netty.util.concurrent.SingleThreadEventExecutor$2 run
WARNING: Unexpected exception from an event executor:
java.lang.OutOfMemoryError: GC overhead limit exceeded

Unless surefire overrides it, all my Maven jobs are assigned 2GB of
heap. I know that's not huge but I prefer it to be conservative to
serve as "canary".

Is that known to be not enough anymore, or worth looking for memory issues?

Thanks,
Sanne


On 3 September 2015 at 14:41, Galder Zamarreno <galder at redhat.com> wrote:
> Hi Sanne,
>
> I've looked at CDI and Compatibility issues, see below.
>
> Cheers,
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
> ----- Original Message -----
>> Hey Sanne! Yep you are right ignoring output is BAD IDEA. I realized that
>> it's difficult to look through all log manually so probably we should write
>> some parser in python or bash to grep it and put it into bin/ folder with
>> other scripts, So at least we can run this script after all tests were run
>> and analyze it somehow. And about "appear to be good" nobody knows why. It
>> could be testng/junit issue as we mix it a lot. So this needs further
>> discussion and analysis.
>>
>> Vitalii
>>
>> ----- Вихідне повідомлення -----
>> Від: "Sanne Grinovero" <sanne at infinispan.org>
>> Кому: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> Надіслано: Понеділок, 10 Серпень 2015 р 20:46:06
>> Тема: [infinispan-dev] Hidden failures in the testsuite
>>
>> Hi all,
>> I just updated my local master fork and started the testsuite, as I
>> sometimes do.
>>
>> It's great to see that the build was successful, and no tests
>> *appeared* to have failed.
>>
>> But! lazily scrolling up in the console, I see lots of exceptions
>> which don't look like intentional (I'm aware that some tests
>> intentionally create error conditions). Also some tests are extremely
>> verbose, which might be the reason for nobody noticing these.
>>
>> Some examples:
>>  - org.infinispan.it.compatibility.EmbeddedRestHotRodTest seems to log
>> TRACE to the console (and probably the whole module)
>
> ^ I've run the compatibility testsuite manually and didn't have such issue with master:
> https://gist.github.com/galderz/b59f1ed4599229022f27
>
> Are you still having issues with this?
>
>>  - CDI tests such as org.infinispan.cdi.InfinispanExtensionRemote seem
>> to fail in great number because of some ClassNotFoundException(s)
>> and/or ResourceLoadingException(s)
>
> ^ Hmmmm, not seeing any of that either:
> https://gist.github.com/galderz/1143078e6be8869cd602
>
> Are you still having issues with this?
>
>>  - OSGi integration tests seem to be all broken by some invalid
>> integration with Aries / Geronimo
>>  - OSGi integration tests dump a lot of unnecessary information to the
>> build console
>>  - the Infinispan Query tests log lots of WARN too, around missing
>> configuration properties and in some cases concerning exceptions; I'm
>> pretty sure that I had resolved those in the past, seems some
>> refactorings were done w/o considering the log outputs.
>>
>> Please don't ignore the output; if it's too verbose to watch, that
>> needs to be resolved too.
>>
>> I also monitor the "expected execution time" of some modules I'm
>> interested in, that's been useful in some cases to figure out that
>> there was some regression.
>>
>> One big question: why is it that so many tests "appear to be good" but
>> are actually broken? I would like to understand that.
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev