2:44 p.m.
On 12/12/16 17:51, Stian Thorgersen wrote:
It should run at least what it used to run before I broke it up into
multiple parallell jobs (seems I did a poor job when I did that and
messed some stuff up). That can be fixed straight away, but adding
more tests (new adapter tests, clustering tests, console tests, etc.)
we should probably review later as it may impact stability and time to
test a PR on Travis.
+1 for better review. Few things:
* Unit tests in the modules outside testsuite are not executed during
travis build. For example tests under core/src/test/java .
* Everything in testsuite/integration and
testsuite/integration-arquillian/tests/base is running in Travis build
ATM. However if we add new package with tests into the testsuite (For
example org.keycloak.testsuite.zzz.NewTest), it is possible that it
won't be running due to missing filter in the travis-run-tests.sh for
the package starting with "z" . I hope we can have the list of packages
by travis-run-tests.sh to be more dynamic.
* IMO it is sufficient that travis will run all the tests (including
adapters and cluster) on embedded undertow. Running the tests on Wildfly
is unecessary overhead, which will slightly increase the travis build
time (as it will need to build distribution, start server etc) . There
is not much added value as 95 % of regressions are catched with undertow
too.
* I've added some undertow adapter tests to be executed during default
travis build. They are all in the package
org.keycloak.testsuite.adapter.undertow.servlet . Problem is, that not
all adapter tests are executed as it will require creating subclass for
every "Abstract" class with test and I just didn't create all the
classes for embedded undertow. I've created JIRA
https://issues.jboss.org/browse/KEYCLOAK-4069 to omprove adapter tests
to avoid "dead" modules and many classes needed for single adapter test.
Marek
On 12 December 2016 at 17:46, Bill Burke <bburke(a)redhat.com
<mailto:bburke@redhat.com>> wrote:
This wasn't just the adapter tests, initially it was the
PartialImportsTest which lives under org.keycloak.testsuite.admin
somewhere and should have run and should have failed.
https://issues.jboss.org/browse/KEYCLOAK-4068
<https://issues.jboss.org/browse/KEYCLOAK-4068>
On 12/12/16 8:24 AM, Stian Thorgersen wrote:
> Let's review the tests ran by Travis after the new year, but with
> Travis having the option to run groups of tests in parallel we
> should be able to tests more including adapters and console.
> Travis should also be changed to tests with the full KC server
> and WildFly for adapters rather than embedded Undertow.
>
> On 5 December 2016 at 16:32, Bill Burke <bburke(a)redhat.com
> <mailto:bburke@redhat.com>> wrote:
>
> These broke because they weren't part of the main build. I
> thought they
> were just dead code because when I did a "Find Usages" for
> them, nothing
> came up. Minimally, things should at least be compiled with
> the main
> build, that way when refactorings happen, somebody doesn't
> delete a test
> dependency by accident.
>
>
> On 12/5/16 6:53 AM, Hynek Mlnarik wrote:
> > Speaking of the tests, we seem not to run any of the
> adapter test
> > suites. I believe that running at least adapter tests for
> wildfly
> > would be beneficial, preventing e.g. [1]. WDYT?
> >
> > [1] https://issues.jboss.org/browse/KEYCLOAK-4017
> <https://issues.jboss.org/browse/KEYCLOAK-4017>
> >
> > On Mon, Dec 5, 2016 at 8:39 AM, Marek Posolda
> <mposolda(a)redhat.com <mailto:mposolda@redhat.com>
> > <mailto:mposolda@redhat.com
<mailto:mposolda@redhat.com>>>
> wrote:
> >
> > Fyi. On Friday afternoon, I've found the issue that
> travis didn't run
> > all the tests from the testsuite. And PartialImportTest
> was the one,
> > which wasn't executed. See
> > https://issues.jboss.org/browse/KEYCLOAK-4021
> <https://issues.jboss.org/browse/KEYCLOAK-4021>
> > <https://issues.jboss.org/browse/KEYCLOAK-4021
> <https://issues.jboss.org/browse/KEYCLOAK-4021>>
> >
> > However with the travis fix, all the tests were passing
> (including
> > PartialImportTest). So it seems this test was just failing
> > randomly (not
> > always)?
> >
> > Now I can see in latest travis build that
> PartialImportTest passes.
> >
> > Marek
> >
> >
> >
> >
> > On 03/12/16 17:36, Bill Burke wrote:
> > > I just noticed that my local build fails while travis
> passes.
> > The bug is
> > > really something travis should have picked up,
> specifically the
> > > PartialImportsTest was removing an identity
> provider. The JPA
> > > removeIdenittyProviderByAlias method was wrong as it
> was trying
> > to load
> > > an IdentityProviderModel after it was removed thus
> resulting in a
> > > Hibernate error. Travis did not pick this up which
> makes me
> > wonder if
> > > the test is even running.
> > >
> > > FYI, i have a pull request that fixes this that is
> incoming. The
> > bug,
> > > not travis.
> > >
> > > Bill
> > >
> > >
> > > _______________________________________________
> > > keycloak-dev mailing list
> > > keycloak-dev(a)lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>
> <mailto:keycloak-dev@lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>>
> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> <https://lists.jboss.org/mailman/listinfo/keycloak-dev>
> > <https://lists.jboss.org/mailman/listinfo/keycloak-dev
> <https://lists.jboss.org/mailman/listinfo/keycloak-dev>>
> >
> >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>
> <mailto:keycloak-dev@lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>>
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> <https://lists.jboss.org/mailman/listinfo/keycloak-dev>
> > <https://lists.jboss.org/mailman/listinfo/keycloak-dev
> <https://lists.jboss.org/mailman/listinfo/keycloak-dev>>
> >
> >
> >
> >
> > --
> >
> > --Hynek
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> <https://lists.jboss.org/mailman/listinfo/keycloak-dev>
>
>