[keycloak-dev] local build failing, travis passing

[Stian Thorgersen]

On 12 December 2016 at 21:44, Marek Posolda <mposolda at redhat.com> wrote:

> On 12/12/16 17:51, Stian Thorgersen wrote:
>
> It should run at least what it used to run before I broke it up into
> multiple parallell jobs (seems I did a poor job when I did that and messed
> some stuff up). That can be fixed straight away, but adding more tests (new
> adapter tests, clustering tests, console tests, etc.) we should probably
> review later as it may impact stability and time to test a PR on Travis.
>
> +1 for better review. Few things:
>
> * Unit tests in the modules outside testsuite are not executed during
> travis build. For example tests under core/src/test/java .
>
> * Everything in testsuite/integration and testsuite/integration-arquillian/tests/base
> is running in Travis build ATM. However if we add new package with tests
> into the testsuite (For example org.keycloak.testsuite.zzz.NewTest), it
> is possible that it won't be running due to missing filter in the
> travis-run-tests.sh for the package starting with "z" . I hope we can have
> the list of packages by travis-run-tests.sh to be more dynamic.
>
> * IMO it is sufficient that travis will run all the tests (including
> adapters and cluster) on embedded undertow. Running the tests on Wildfly is
> unecessary overhead, which will slightly increase the travis build time (as
> it will need to build distribution, start server etc) . There is not much
> added value as 95 % of regressions are catched with undertow too.
>

-1 We've seen this many times where Travis is happy, but the distribution
is broken. This happened several times last year so we need to prevent this.


>
> * I've added some undertow adapter tests to be executed during default
> travis build. They are all in the package org.keycloak.testsuite.
> adapter.undertow.servlet . Problem is, that not all adapter tests are
> executed as it will require creating subclass for every "Abstract" class
> with test and I just didn't create all the classes for embedded undertow.
> I've created JIRA https://issues.jboss.org/browse/KEYCLOAK-4069 to
> omprove adapter tests to avoid "dead" modules and many classes needed for
> single adapter test.
>
> Marek
>
> On 12 December 2016 at 17:46, Bill Burke <bburke at redhat.com> wrote:
>
>> This wasn't just the adapter tests, initially it was the
>> PartialImportsTest which lives under org.keycloak.testsuite.admin somewhere
>> and should have run and should have failed.
>>
>> https://issues.jboss.org/browse/KEYCLOAK-4068
>>
>>
>> On 12/12/16 8:24 AM, Stian Thorgersen wrote:
>>
>> Let's review the tests ran by Travis after the new year, but with Travis
>> having the option to run groups of tests in parallel we should be able to
>> tests more including adapters and console. Travis should also be changed to
>> tests with the full KC server and WildFly for adapters rather than embedded
>> Undertow.
>>
>> On 5 December 2016 at 16:32, Bill Burke <bburke at redhat.com> wrote:
>>
>>> These broke because they weren't part of the main build.  I thought they
>>> were just dead code because when I did a "Find Usages" for them, nothing
>>> came up.  Minimally, things should at least be compiled with the main
>>> build, that way when refactorings happen, somebody doesn't delete a test
>>> dependency by accident.
>>>
>>>
>>> On 12/5/16 6:53 AM, Hynek Mlnarik wrote:
>>> > Speaking of the tests, we seem not to run any of the adapter test
>>> > suites. I believe that running at least adapter tests for wildfly
>>> > would be beneficial, preventing e.g. [1]. WDYT?
>>> >
>>> > [1] https://issues.jboss.org/browse/KEYCLOAK-4017
>>> >
>>> > On Mon, Dec 5, 2016 at 8:39 AM, Marek Posolda <mposolda at redhat.com
>>> > <mailto:mposolda at redhat.com>> wrote:
>>> >
>>> >     Fyi. On Friday afternoon, I've found the issue that travis didn't
>>> run
>>> >     all the tests from the testsuite. And PartialImportTest was the
>>> one,
>>> >     which wasn't executed. See
>>> >     https://issues.jboss.org/browse/KEYCLOAK-4021
>>> >     <https://issues.jboss.org/browse/KEYCLOAK-4021>
>>> >
>>> >     However with the travis fix, all the tests were passing (including
>>> >     PartialImportTest). So it seems this test was just failing
>>> >     randomly (not
>>> >     always)?
>>> >
>>> >     Now I can see in latest travis build that PartialImportTest passes.
>>> >
>>> >     Marek
>>> >
>>> >
>>> >
>>> >
>>> >     On 03/12/16 17:36, Bill Burke wrote:
>>> >     > I just noticed that my local build fails while travis passes.
>>> >     The bug is
>>> >     > really something travis should have picked up, specifically the
>>> >     > PartialImportsTest was removing an identity provider.  The JPA
>>> >     > removeIdenittyProviderByAlias method was wrong as it was trying
>>> >     to load
>>> >     > an IdentityProviderModel after it was removed thus resulting in a
>>> >     > Hibernate error.  Travis did not pick this up which makes me
>>> >     wonder if
>>> >     > the test is even running.
>>> >     >
>>> >     > FYI, i have a pull request that fixes this that is incoming. The
>>> >     bug,
>>> >     > not travis.
>>> >     >
>>> >     > Bill
>>> >     >
>>> >     >
>>> >     > _______________________________________________
>>> >     > keycloak-dev mailing list
>>> >     > keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jbo
>>> ss.org>
>>> >     > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> >     <https://lists.jboss.org/mailman/listinfo/keycloak-dev>
>>> >
>>> >
>>> >     _______________________________________________
>>> >     keycloak-dev mailing list
>>> >     keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>>> >     https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> >     <https://lists.jboss.org/mailman/listinfo/keycloak-dev>
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> >
>>> > --Hynek
>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>
>>
>>
>
>