[keycloak-dev] All PR's failing

Fri Aug 9 08:34:12 EDT 2019

+1

It seems like things are back to normal on Travis
(https://www.traviscistatus.com/). Let's see how it goes.

On Fri, Aug 9, 2019 at 9:23 AM Pedro Igor Silva <psilva at redhat.com> wrote:
>
>
>
> On Fri, Aug 9, 2019 at 5:25 AM Marek Posolda <mposolda at redhat.com> wrote:
>>
>> On 09. 08. 19 9:45, Sebastian Laskawiec wrote:
>> >
>> >
>> > On Thu, Aug 8, 2019 at 5:32 PM Bruno Oliveira <bruno at abstractj.org
>> > <mailto:bruno at abstractj.org>> wrote:
>> >
>> >     I believe Hynek's changes help us a lot with the PRs and solves our
>> >     problem for now. Yesterday after merging Stan's PR which only touches
>> >     into non-Java files, the build from master started to fail for the
>> >     cross-dc Job.
>> >
>> >     Cross DC builds as Authz builds run conditionally against
>> >     pull-request,
>> >     not against master. Here is my suggestion:
>> >
>> >     1. For these builds that have been consistently failing. We apply the
>> >     same conditions that we have for pull-request for master. Into other
>> >     words, if I didn't touch into cross-dc stuff, I don't run any tests
>> >     related to this when my changes are merged.
>> >
>> >
>> > Is there any benefit on implementing this conditional runs? Perhaps we
>> > should always run the whole test suite. This way we could discover
>> > unstable tests faster and hopefully get them fixed.
>>
>> For cross-dc tests, the problem is that they are not so stable. Running
>> it always will decrease travis stability and in most cases, running them
>> is not really necessarily. But as you pointed, there is possible issue
>> that we won't discover issues early and cross-dc tests can become broken
>> even with changes, which doesn't seem to be related to cross-dc (but
>> that happen only once during last 1.5 years since we have cross-dc
>> tests). So there are both pros and cons, but I vote to rather keep
>> cross-dc tests in travis conditional.
>>
>> For authz tests, I am not sure. If they are stable, we can run them
>> always in theory. But I am not sure if they are really stable?
>
>
> They should be stable. In the adapter tests, the most critical one is using angular, but after some changes from Michal (I think) the tests were running just fine.
>
> At least for my PRs, the errors I usually get are related with maven deps not being fetched or cross-dc.
>
>>
>>
>> Marek
>>
>> >
>> >     2. We have a Daily Job running on Travis against master. Into our
>> >     scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
>> >     "cron", we run all the tests: cross-dc, authz and others.
>> >
>> >
>> >     If that makes sense and we have some agreement about this. I can
>> >     give it
>> >     a try.
>> >
>> >     On 2019-08-05, Marek Posolda wrote:
>> >     > On 05. 08. 19 19:13, Stan Silvert wrote:
>> >     > > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
>> >     > > > The PRs are blocked by two issues:
>> >     > > > 1) Adapter tests sometimes take too much time: This should be
>> >     > > > addressed by (now merged) [1]
>> >     > > > 2) OpenShiftTokenReviewEndpointTest being unstable,
>> >     addressed by [2]
>> >     > > >
>> >     > > > IIRC, we had issues with Travis caches [3, 4], but if there
>> >     is time it
>> >     > > > might be worth checking whether the issues persist.
>> >     > > >
>> >     > > > Once [2] is merged, Travis should be stable again for
>> >     rebased PRs.
>> >     > > Do we know when this is going to happen?
>> >     >
>> >     > PR is approved by Hynek and me, but travis didn't pass for the
>> >     PR due the
>> >     > adapter tests exceed the 50 minutes limit. I am retrying one
>> >     more time.
>> >     >
>> >     > I agree with what Bruno's suggested to split adapter tests into
>> >     more groups.
>> >     > Or fix adapter tests to not require realm re-import after each
>> >     test as that
>> >     > take significant amount of time. See [1]. But that will probably
>> >     require
>> >     > some more changes and may be probably few days of work...
>> >     >
>> >     > [1]
>> >     https://github.com/keycloak/keycloak/blob/master/testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/adapter/AbstractAdapterTest.java#L124-L128
>> >     >
>> >     > Marek
>> >     >
>> >     >
>> >     > >
>> >     > > > --Hynek
>> >     > > >
>> >     > > > [1] https://github.com/keycloak/keycloak/pull/6209
>> >     > > > [2] https://github.com/keycloak/keycloak/pull/6207
>> >     > > > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
>> >     > > > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
>> >     > > >
>> >     > > > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira
>> >     <bruno at abstractj.org <mailto:bruno at abstractj.org>
>> >     > > > <mailto:bruno at abstractj.org <mailto:bruno at abstractj.org>>>
>> >     wrote:
>> >     > > >
>> >     > > >      Before answer this e-mail I was doing some experiments
>> >     and there are
>> >     > > >      some things that I identified we can do about this.
>> >     > > >
>> >     > > >      1. Enable cache again on Travis
>> >     > > >
>> >     > > >      That won't fix the issue, but may help
>> >     > > >
>> >     > > >      2. Split the adapter tests
>> >     > > >
>> >     > > >      I did some test with the following PR:
>> >     > > > https://github.com/keycloak/keycloak/pull/6221/files
>> >     > > >
>> >     > > >      Here's the output:
>> >     > > > https://travis-ci.org/keycloak/keycloak/builds/567252894
>> >     > > >
>> >     > > >      3. Review the tests we run on Travis
>> >     > > >
>> >     > > >      I'm not so sure if everything that we have running on
>> >     Travis today is
>> >     > > >      really required.
>> >     > > >
>> >     > > >
>> >     > > >      For now if others agree on item 2, I can polish my PR
>> >     and work to
>> >     > > >      get it merged.
>> >     > > >
>> >     > > >      On 2019-08-02, Stan Silvert wrote:
>> >     > > >      > The adapter tests are timing out on all new PR's.
>> >     > > >      >
>> >     > > >      > Hynek, I'm told that you might be able to help?
>> >     > > >      >
>> >     > > >      > Stan
>> >     > > >      >
>> >     > > >      > _______________________________________________
>> >     > > >      > keycloak-dev mailing list
>> >     > > >      > keycloak-dev at lists.jboss.org
>> >     <mailto:keycloak-dev at lists.jboss.org>
>> >     <mailto:keycloak-dev at lists.jboss.org
>> >     <mailto:keycloak-dev at lists.jboss.org>>
>> >     > > >      > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> >     > > >
>> >     > > >      --
>> >     > > >
>> >     > > >      abstractj
>> >     > > >
>> >     > > _______________________________________________
>> >     > > keycloak-dev mailing list
>> >     > > keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>> >     > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> >     >
>> >     >
>> >
>> >     --
>> >
>> >     abstractj
>> >     _______________________________________________
>> >     keycloak-dev mailing list
>> >     keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>> >     https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> >
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev

-- 
- abstractj