[keycloak-dev] All PR's failing
Marek Posolda
mposolda at redhat.com
Fri Aug 9 04:19:43 EDT 2019
On 09. 08. 19 9:45, Sebastian Laskawiec wrote:
>
>
> On Thu, Aug 8, 2019 at 5:32 PM Bruno Oliveira <bruno at abstractj.org
> <mailto:bruno at abstractj.org>> wrote:
>
> I believe Hynek's changes help us a lot with the PRs and solves our
> problem for now. Yesterday after merging Stan's PR which only touches
> into non-Java files, the build from master started to fail for the
> cross-dc Job.
>
> Cross DC builds as Authz builds run conditionally against
> pull-request,
> not against master. Here is my suggestion:
>
> 1. For these builds that have been consistently failing. We apply the
> same conditions that we have for pull-request for master. Into other
> words, if I didn't touch into cross-dc stuff, I don't run any tests
> related to this when my changes are merged.
>
>
> Is there any benefit on implementing this conditional runs? Perhaps we
> should always run the whole test suite. This way we could discover
> unstable tests faster and hopefully get them fixed.
For cross-dc tests, the problem is that they are not so stable. Running
it always will decrease travis stability and in most cases, running them
is not really necessarily. But as you pointed, there is possible issue
that we won't discover issues early and cross-dc tests can become broken
even with changes, which doesn't seem to be related to cross-dc (but
that happen only once during last 1.5 years since we have cross-dc
tests). So there are both pros and cons, but I vote to rather keep
cross-dc tests in travis conditional.
For authz tests, I am not sure. If they are stable, we can run them
always in theory. But I am not sure if they are really stable?
Marek
>
> 2. We have a Daily Job running on Travis against master. Into our
> scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
> "cron", we run all the tests: cross-dc, authz and others.
>
>
> If that makes sense and we have some agreement about this. I can
> give it
> a try.
>
> On 2019-08-05, Marek Posolda wrote:
> > On 05. 08. 19 19:13, Stan Silvert wrote:
> > > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> > > > The PRs are blocked by two issues:
> > > > 1) Adapter tests sometimes take too much time: This should be
> > > > addressed by (now merged) [1]
> > > > 2) OpenShiftTokenReviewEndpointTest being unstable,
> addressed by [2]
> > > >
> > > > IIRC, we had issues with Travis caches [3, 4], but if there
> is time it
> > > > might be worth checking whether the issues persist.
> > > >
> > > > Once [2] is merged, Travis should be stable again for
> rebased PRs.
> > > Do we know when this is going to happen?
> >
> > PR is approved by Hynek and me, but travis didn't pass for the
> PR due the
> > adapter tests exceed the 50 minutes limit. I am retrying one
> more time.
> >
> > I agree with what Bruno's suggested to split adapter tests into
> more groups.
> > Or fix adapter tests to not require realm re-import after each
> test as that
> > take significant amount of time. See [1]. But that will probably
> require
> > some more changes and may be probably few days of work...
> >
> > [1]
> https://github.com/keycloak/keycloak/blob/master/testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/adapter/AbstractAdapterTest.java#L124-L128
> >
> > Marek
> >
> >
> > >
> > > > --Hynek
> > > >
> > > > [1] https://github.com/keycloak/keycloak/pull/6209
> > > > [2] https://github.com/keycloak/keycloak/pull/6207
> > > > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> > > > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> > > >
> > > > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira
> <bruno at abstractj.org <mailto:bruno at abstractj.org>
> > > > <mailto:bruno at abstractj.org <mailto:bruno at abstractj.org>>>
> wrote:
> > > >
> > > > Before answer this e-mail I was doing some experiments
> and there are
> > > > some things that I identified we can do about this.
> > > >
> > > > 1. Enable cache again on Travis
> > > >
> > > > That won't fix the issue, but may help
> > > >
> > > > 2. Split the adapter tests
> > > >
> > > > I did some test with the following PR:
> > > > https://github.com/keycloak/keycloak/pull/6221/files
> > > >
> > > > Here's the output:
> > > > https://travis-ci.org/keycloak/keycloak/builds/567252894
> > > >
> > > > 3. Review the tests we run on Travis
> > > >
> > > > I'm not so sure if everything that we have running on
> Travis today is
> > > > really required.
> > > >
> > > >
> > > > For now if others agree on item 2, I can polish my PR
> and work to
> > > > get it merged.
> > > >
> > > > On 2019-08-02, Stan Silvert wrote:
> > > > > The adapter tests are timing out on all new PR's.
> > > > >
> > > > > Hynek, I'm told that you might be able to help?
> > > > >
> > > > > Stan
> > > > >
> > > > > _______________________________________________
> > > > > keycloak-dev mailing list
> > > > > keycloak-dev at lists.jboss.org
> <mailto:keycloak-dev at lists.jboss.org>
> <mailto:keycloak-dev at lists.jboss.org
> <mailto:keycloak-dev at lists.jboss.org>>
> > > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > > >
> > > > --
> > > >
> > > > abstractj
> > > >
> > > _______________________________________________
> > > keycloak-dev mailing list
> > > keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> >
>
> --
>
> abstractj
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
More information about the keycloak-dev
mailing list