[keycloak-dev] All PR's failing

[Sebastian Laskawiec]

On Thu, Aug 8, 2019 at 5:32 PM Bruno Oliveira <bruno at abstractj.org> wrote:

> I believe Hynek's changes help us a lot with the PRs and solves our
> problem for now. Yesterday after merging Stan's PR which only touches
> into non-Java files, the build from master started to fail for the
> cross-dc Job.
>
> Cross DC builds as Authz builds run conditionally against pull-request,
> not against master. Here is my suggestion:
>
> 1. For these builds that have been consistently failing. We apply the
> same conditions that we have for pull-request for master. Into other
> words, if I didn't touch into cross-dc stuff, I don't run any tests
> related to this when my changes are merged.
>

Is there any benefit on implementing this conditional runs? Perhaps we
should always run the whole test suite. This way we could discover unstable
tests faster and hopefully get them fixed.


>
> 2. We have a Daily Job running on Travis against master. Into our
> scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
> "cron", we run all the tests: cross-dc, authz and others.
>

> If that makes sense and we have some agreement about this. I can give it
> a try.
>
> On 2019-08-05, Marek Posolda wrote:
> > On 05. 08. 19 19:13, Stan Silvert wrote:
> > > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> > > > The PRs are blocked by two issues:
> > > > 1) Adapter tests sometimes take too much time: This should be
> > > > addressed by (now merged) [1]
> > > > 2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
> > > >
> > > > IIRC, we had issues with Travis caches [3, 4], but if there is time
> it
> > > > might be worth checking whether the issues persist.
> > > >
> > > > Once [2] is merged, Travis should be stable again for rebased PRs.
> > > Do we know when this is going to happen?
> >
> > PR is approved by Hynek and me, but travis didn't pass for the PR due the
> > adapter tests exceed the 50 minutes limit. I am retrying one more time.
> >
> > I agree with what Bruno's suggested to split adapter tests into more
> groups.
> > Or fix adapter tests to not require realm re-import after each test as
> that
> > take significant amount of time. See [1]. But that will probably require
> > some more changes and may be probably few days of work...
> >
> > [1]
> https://github.com/keycloak/keycloak/blob/master/testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/adapter/AbstractAdapterTest.java#L124-L128
> >
> > Marek
> >
> >
> > >
> > > > --Hynek
> > > >
> > > > [1] https://github.com/keycloak/keycloak/pull/6209
> > > > [2] https://github.com/keycloak/keycloak/pull/6207
> > > > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> > > > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> > > >
> > > > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno at abstractj.org
> > > > <mailto:bruno at abstractj.org>> wrote:
> > > >
> > > >      Before answer this e-mail I was doing some experiments and
> there are
> > > >      some things that I identified we can do about this.
> > > >
> > > >      1. Enable cache again on Travis
> > > >
> > > >      That won't fix the issue, but may help
> > > >
> > > >      2. Split the adapter tests
> > > >
> > > >      I did some test with the following PR:
> > > >      https://github.com/keycloak/keycloak/pull/6221/files
> > > >
> > > >      Here's the output:
> > > >      https://travis-ci.org/keycloak/keycloak/builds/567252894
> > > >
> > > >      3. Review the tests we run on Travis
> > > >
> > > >      I'm not so sure if everything that we have running on Travis
> today is
> > > >      really required.
> > > >
> > > >
> > > >      For now if others agree on item 2, I can polish my PR and work
> to
> > > >      get it merged.
> > > >
> > > >      On 2019-08-02, Stan Silvert wrote:
> > > >      > The adapter tests are timing out on all new PR's.
> > > >      >
> > > >      > Hynek, I'm told that you might be able to help?
> > > >      >
> > > >      > Stan
> > > >      >
> > > >      > _______________________________________________
> > > >      > keycloak-dev mailing list
> > > >      > keycloak-dev at lists.jboss.org <mailto:
> keycloak-dev at lists.jboss.org>
> > > >      > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > > >
> > > >      --
> > > >
> > > >      abstractj
> > > >
> > > _______________________________________________
> > > keycloak-dev mailing list
> > > keycloak-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> >
>
> --
>
> abstractj
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>