On 12/12/16 17:51, Stian Thorgersen wrote:
It should run at least what it used to run before I broke it up into
multiple parallell jobs (seems I did a poor job when I did that and messed
some stuff up). That can be fixed straight away, but adding more tests (new
adapter tests, clustering tests, console tests, etc.) we should probably
review later as it may impact stability and time to test a PR on Travis.
+1 for better review. Few things:
* Unit tests in the modules outside testsuite are not executed during
travis build. For example tests under core/src/test/java .
* Everything in testsuite/integration and testsuite/integration-arquillian/tests/base
is running in Travis build ATM. However if we add new package with tests
into the testsuite (For example org.keycloak.testsuite.zzz.NewTest), it
is possible that it won't be running due to missing filter in the
travis-run-tests.sh for the package starting with "z" . I hope we can have
the list of packages by travis-run-tests.sh to be more dynamic.
* IMO it is sufficient that travis will run all the tests (including
adapters and cluster) on embedded undertow. Running the tests on Wildfly is
unecessary overhead, which will slightly increase the travis build time (as
it will need to build distribution, start server etc) . There is not much
added value as 95 % of regressions are catched with undertow too.
-1 We've seen this many times where Travis is happy, but the distribution
is broken. This happened several times last year so we need to prevent this.
* I've added some undertow adapter tests to be executed during default
travis build. They are all in the package org.keycloak.testsuite.
adapter.undertow.servlet . Problem is, that not all adapter tests are
executed as it will require creating subclass for every "Abstract" class
with test and I just didn't create all the classes for embedded undertow.
I've created JIRA
https://issues.jboss.org/browse/KEYCLOAK-4069 to
omprove adapter tests to avoid "dead" modules and many classes needed for
single adapter test.
Marek
On 12 December 2016 at 17:46, Bill Burke <bburke(a)redhat.com> wrote:
> This wasn't just the adapter tests, initially it was the
> PartialImportsTest which lives under org.keycloak.testsuite.admin somewhere
> and should have run and should have failed.
>
>
https://issues.jboss.org/browse/KEYCLOAK-4068
>
>
> On 12/12/16 8:24 AM, Stian Thorgersen wrote:
>
> Let's review the tests ran by Travis after the new year, but with Travis
> having the option to run groups of tests in parallel we should be able to
> tests more including adapters and console. Travis should also be changed to
> tests with the full KC server and WildFly for adapters rather than embedded
> Undertow.
>
> On 5 December 2016 at 16:32, Bill Burke <bburke(a)redhat.com> wrote:
>
>> These broke because they weren't part of the main build. I thought they
>> were just dead code because when I did a "Find Usages" for them,
nothing
>> came up. Minimally, things should at least be compiled with the main
>> build, that way when refactorings happen, somebody doesn't delete a test
>> dependency by accident.
>>
>>
>> On 12/5/16 6:53 AM, Hynek Mlnarik wrote:
>> > Speaking of the tests, we seem not to run any of the adapter test
>> > suites. I believe that running at least adapter tests for wildfly
>> > would be beneficial, preventing e.g. [1]. WDYT?
>> >
>> > [1]
https://issues.jboss.org/browse/KEYCLOAK-4017
>> >
>> > On Mon, Dec 5, 2016 at 8:39 AM, Marek Posolda <mposolda(a)redhat.com
>> > <mailto:mposolda@redhat.com>> wrote:
>> >
>> > Fyi. On Friday afternoon, I've found the issue that travis
didn't
>> run
>> > all the tests from the testsuite. And PartialImportTest was the
>> one,
>> > which wasn't executed. See
>> >
https://issues.jboss.org/browse/KEYCLOAK-4021
>> > <
https://issues.jboss.org/browse/KEYCLOAK-4021>
>> >
>> > However with the travis fix, all the tests were passing (including
>> > PartialImportTest). So it seems this test was just failing
>> > randomly (not
>> > always)?
>> >
>> > Now I can see in latest travis build that PartialImportTest passes.
>> >
>> > Marek
>> >
>> >
>> >
>> >
>> > On 03/12/16 17:36, Bill Burke wrote:
>> > > I just noticed that my local build fails while travis passes.
>> > The bug is
>> > > really something travis should have picked up, specifically the
>> > > PartialImportsTest was removing an identity provider. The JPA
>> > > removeIdenittyProviderByAlias method was wrong as it was trying
>> > to load
>> > > an IdentityProviderModel after it was removed thus resulting in a
>> > > Hibernate error. Travis did not pick this up which makes me
>> > wonder if
>> > > the test is even running.
>> > >
>> > > FYI, i have a pull request that fixes this that is incoming. The
>> > bug,
>> > > not travis.
>> > >
>> > > Bill
>> > >
>> > >
>> > > _______________________________________________
>> > > keycloak-dev mailing list
>> > > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jbo
>> ss.org>
>> > >
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> > <
https://lists.jboss.org/mailman/listinfo/keycloak-dev>
>> >
>> >
>> > _______________________________________________
>> > keycloak-dev mailing list
>> > keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
>> >
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> > <
https://lists.jboss.org/mailman/listinfo/keycloak-dev>
>> >
>> >
>> >
>> >
>> > --
>> >
>> > --Hynek
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>
>
>