12:48 p.m.
The adapter tests are timing out on all new PR's.
Hynek, I'm told that you might be able to help?
Stan
5:52 a.m.
Before answer this e-mail I was doing some experiments and there are
some things that I identified we can do about this.
1. Enable cache again on Travis
That won't fix the issue, but may help
2. Split the adapter tests
I did some test with the following PR:
https://github.com/keycloak/keycloak/pull/6221/files
Here's the output:
https://travis-ci.org/keycloak/keycloak/builds/567252894
3. Review the tests we run on Travis
I'm not so sure if everything that we have running on Travis today is
really required.
For now if others agree on item 2, I can polish my PR and work to get it merged.
On 2019-08-02, Stan Silvert wrote:
The adapter tests are timing out on all new PR's.
Hynek, I'm told that you might be able to help?
Stan
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
abstractj
2:17 a.m.
On Sat, Aug 3, 2019 at 1:03 PM Bruno Oliveira <bruno(a)abstractj.org> wrote:
Before answer this e-mail I was doing some experiments and there are
some things that I identified we can do about this.
1. Enable cache again on Travis
That won't fix the issue, but may help
I've been thinking about this for a while.... We might enable cache on
~/.m2/repository and wipe out ~/.m2/repository/org/keycloak on every Travis
run.
WDYT?
2. Split the adapter tests
I did some test with the following PR:
https://github.com/keycloak/keycloak/pull/6221/files
Here's the output:
https://travis-ci.org/keycloak/keycloak/builds/567252894
3. Review the tests we run on Travis
I'm not so sure if everything that we have running on Travis today is
really required.
For now if others agree on item 2, I can polish my PR and work to get it
merged.
On 2019-08-02, Stan Silvert wrote:
> The adapter tests are timing out on all new PR's.
>
> Hynek, I'm told that you might be able to help?
>
> Stan
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
abstractj
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
2:43 a.m.
The PRs are blocked by two issues:
1) Adapter tests sometimes take too much time: This should be addressed by
(now merged) [1]
2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
IIRC, we had issues with Travis caches [3, 4], but if there is time it
might be worth checking whether the issues persist.
Once [2] is merged, Travis should be stable again for rebased PRs.
--Hynek
[1] https://github.com/keycloak/keycloak/pull/6209
[2] https://github.com/keycloak/keycloak/pull/6207
[3] https://issues.jboss.org/browse/KEYCLOAK-5124
[4] https://issues.jboss.org/browse/KEYCLOAK-7285
On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org> wrote:
Before answer this e-mail I was doing some experiments and there are
some things that I identified we can do about this.
1. Enable cache again on Travis
That won't fix the issue, but may help
2. Split the adapter tests
I did some test with the following PR:
https://github.com/keycloak/keycloak/pull/6221/files
Here's the output:
https://travis-ci.org/keycloak/keycloak/builds/567252894
3. Review the tests we run on Travis
I'm not so sure if everything that we have running on Travis today is
really required.
For now if others agree on item 2, I can polish my PR and work to get it
merged.
On 2019-08-02, Stan Silvert wrote:
> The adapter tests are timing out on all new PR's.
>
> Hynek, I'm told that you might be able to help?
>
> Stan
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
abstractj
12:13 p.m.
On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
The PRs are blocked by two issues:
1) Adapter tests sometimes take too much time: This should be
addressed by (now merged) [1]
2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
IIRC, we had issues with Travis caches [3, 4], but if there is time it
might be worth checking whether the issues persist.
Once [2] is merged, Travis should be stable again for rebased PRs.
Do we know when
this is going to happen?
--Hynek
[1] https://github.com/keycloak/keycloak/pull/6209
[2] https://github.com/keycloak/keycloak/pull/6207
[3] https://issues.jboss.org/browse/KEYCLOAK-5124
[4] https://issues.jboss.org/browse/KEYCLOAK-7285
On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org
<mailto:bruno@abstractj.org>> wrote:
Before answer this e-mail I was doing some experiments and there are
some things that I identified we can do about this.
1. Enable cache again on Travis
That won't fix the issue, but may help
2. Split the adapter tests
I did some test with the following PR:
https://github.com/keycloak/keycloak/pull/6221/files
Here's the output:
https://travis-ci.org/keycloak/keycloak/builds/567252894
3. Review the tests we run on Travis
I'm not so sure if everything that we have running on Travis today is
really required.
For now if others agree on item 2, I can polish my PR and work to
get it merged.
On 2019-08-02, Stan Silvert wrote:
> The adapter tests are timing out on all new PR's.
>
> Hynek, I'm told that you might be able to help?
>
> Stan
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
abstractj
12:35 p.m.
Adapter tests are still broken:
https://travis-ci.org/keycloak/keycloak/jobs/567926585
Maybe worth to temporarily split the adapter tests and see if the
situation improves.
On Mon, Aug 5, 2019 at 2:13 PM Stan Silvert <ssilvert(a)redhat.com> wrote:
On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
The PRs are blocked by two issues:
1) Adapter tests sometimes take too much time: This should be addressed by (now merged)
[1]
2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
IIRC, we had issues with Travis caches [3, 4], but if there is time it might be worth
checking whether the issues persist.
Once [2] is merged, Travis should be stable again for rebased PRs.
Do we know when this is going to happen?
--Hynek
[1] https://github.com/keycloak/keycloak/pull/6209
[2] https://github.com/keycloak/keycloak/pull/6207
[3] https://issues.jboss.org/browse/KEYCLOAK-5124
[4] https://issues.jboss.org/browse/KEYCLOAK-7285
On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org> wrote:
>
> Before answer this e-mail I was doing some experiments and there are
> some things that I identified we can do about this.
>
> 1. Enable cache again on Travis
>
> That won't fix the issue, but may help
>
> 2. Split the adapter tests
>
> I did some test with the following PR:
> https://github.com/keycloak/keycloak/pull/6221/files
>
> Here's the output:
> https://travis-ci.org/keycloak/keycloak/builds/567252894
>
> 3. Review the tests we run on Travis
>
> I'm not so sure if everything that we have running on Travis today is
> really required.
>
>
> For now if others agree on item 2, I can polish my PR and work to get it merged.
>
> On 2019-08-02, Stan Silvert wrote:
> > The adapter tests are timing out on all new PR's.
> >
> > Hynek, I'm told that you might be able to help?
> >
> > Stan
> >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
> --
>
> abstractj
--
- abstractj
3:38 p.m.
On 05. 08. 19 19:13, Stan Silvert wrote:
On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> The PRs are blocked by two issues:
> 1) Adapter tests sometimes take too much time: This should be
> addressed by (now merged) [1]
> 2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
>
> IIRC, we had issues with Travis caches [3, 4], but if there is time it
> might be worth checking whether the issues persist.
>
> Once [2] is merged, Travis should be stable again for rebased PRs.
Do we know when this is going to happen?
PR is approved by Hynek and me, but travis didn't pass for the PR due
the adapter tests exceed the 50 minutes limit. I am retrying one more time.
I agree with what Bruno's suggested to split adapter tests into more
groups. Or fix adapter tests to not require realm re-import after each
test as that take significant amount of time. See [1]. But that will
probably require some more changes and may be probably few days of work...
[1]
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
Marek
> --Hynek
>
> [1] https://github.com/keycloak/keycloak/pull/6209
> [2] https://github.com/keycloak/keycloak/pull/6207
> [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> [4] https://issues.jboss.org/browse/KEYCLOAK-7285
>
> On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org
> <mailto:bruno@abstractj.org>> wrote:
>
> Before answer this e-mail I was doing some experiments and there are
> some things that I identified we can do about this.
>
> 1. Enable cache again on Travis
>
> That won't fix the issue, but may help
>
> 2. Split the adapter tests
>
> I did some test with the following PR:
> https://github.com/keycloak/keycloak/pull/6221/files
>
> Here's the output:
> https://travis-ci.org/keycloak/keycloak/builds/567252894
>
> 3. Review the tests we run on Travis
>
> I'm not so sure if everything that we have running on Travis today is
> really required.
>
>
> For now if others agree on item 2, I can polish my PR and work to
> get it merged.
>
> On 2019-08-02, Stan Silvert wrote:
> > The adapter tests are timing out on all new PR's.
> >
> > Hynek, I'm told that you might be able to help?
> >
> > Stan
> >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
> --
>
> abstractj
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
2:56 a.m.
Travis seems inconsistent re the times so better safe than sorry. Hence
I've reopened Bruno's PR [1] for splitting adapter tests, but at the same
time, I'd rather excluded authorization tests completely since they take
most of the time of adapter tests, and only included those when appropriate
[2].
However I'm not convinced that any of these PRs would help. I have just
cancelled a build stuck in the very early stage of the build for almost 20
minutes (git clone, specifically "remote: Counting objects: 100%
(14222/14222), done." before build) which points to infrastructure rather
than testsuite issue.
--Hynek
[1] https://github.com/keycloak/keycloak/pull/6221
[2] https://github.com/keycloak/keycloak/pull/6222
On Mon, Aug 5, 2019 at 10:38 PM Marek Posolda <mposolda(a)redhat.com> wrote:
On 05. 08. 19 19:13, Stan Silvert wrote:
> On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
>> The PRs are blocked by two issues:
>> 1) Adapter tests sometimes take too much time: This should be
>> addressed by (now merged) [1]
>> 2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
>>
>> IIRC, we had issues with Travis caches [3, 4], but if there is time it
>> might be worth checking whether the issues persist.
>>
>> Once [2] is merged, Travis should be stable again for rebased PRs.
> Do we know when this is going to happen?
PR is approved by Hynek and me, but travis didn't pass for the PR due
the adapter tests exceed the 50 minutes limit. I am retrying one more time.
I agree with what Bruno's suggested to split adapter tests into more
groups. Or fix adapter tests to not require realm re-import after each
test as that take significant amount of time. See [1]. But that will
probably require some more changes and may be probably few days of work...
[1]
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
Marek
>
>> --Hynek
>>
>> [1] https://github.com/keycloak/keycloak/pull/6209
>> [2] https://github.com/keycloak/keycloak/pull/6207
>> [3] https://issues.jboss.org/browse/KEYCLOAK-5124
>> [4] https://issues.jboss.org/browse/KEYCLOAK-7285
>>
>> On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org
>> <mailto:bruno@abstractj.org>> wrote:
>>
>> Before answer this e-mail I was doing some experiments and there
are
>> some things that I identified we can do about this.
>>
>> 1. Enable cache again on Travis
>>
>> That won't fix the issue, but may help
>>
>> 2. Split the adapter tests
>>
>> I did some test with the following PR:
>> https://github.com/keycloak/keycloak/pull/6221/files
>>
>> Here's the output:
>> https://travis-ci.org/keycloak/keycloak/builds/567252894
>>
>> 3. Review the tests we run on Travis
>>
>> I'm not so sure if everything that we have running on Travis today
is
>> really required.
>>
>>
>> For now if others agree on item 2, I can polish my PR and work to
>> get it merged.
>>
>> On 2019-08-02, Stan Silvert wrote:
>> > The adapter tests are timing out on all new PR's.
>> >
>> > Hynek, I'm told that you might be able to help?
>> >
>> > Stan
>> >
>> > _______________________________________________
>> > keycloak-dev mailing list
>> > keycloak-dev(a)lists.jboss.org <mailto:
keycloak-dev(a)lists.jboss.org>
>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>> --
>>
>> abstractj
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
3:39 a.m.
Requested Travis assistance as well [1]. There are now several timeouts
caused by compilation of keycloak taking too long, e.g. [2]
[1] https://travis-ci.community/t/travis-builds-fail-due-to-slowness/4571
[2] https://travis-ci.org/keycloak/keycloak/jobs/568226589
On Tue, Aug 6, 2019 at 9:56 AM Hynek Mlnarik <hmlnarik(a)redhat.com> wrote:
Travis seems inconsistent re the times so better safe than sorry.
Hence
I've reopened Bruno's PR [1] for splitting adapter tests, but at the same
time, I'd rather excluded authorization tests completely since they take
most of the time of adapter tests, and only included those when appropriate
[2].
However I'm not convinced that any of these PRs would help. I have just
cancelled a build stuck in the very early stage of the build for almost 20
minutes (git clone, specifically "remote: Counting objects: 100%
(14222/14222), done." before build) which points to infrastructure rather
than testsuite issue.
--Hynek
[1] https://github.com/keycloak/keycloak/pull/6221
[2] https://github.com/keycloak/keycloak/pull/6222
On Mon, Aug 5, 2019 at 10:38 PM Marek Posolda <mposolda(a)redhat.com> wrote:
> On 05. 08. 19 19:13, Stan Silvert wrote:
> > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> >> The PRs are blocked by two issues:
> >> 1) Adapter tests sometimes take too much time: This should be
> >> addressed by (now merged) [1]
> >> 2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
> >>
> >> IIRC, we had issues with Travis caches [3, 4], but if there is time it
> >> might be worth checking whether the issues persist.
> >>
> >> Once [2] is merged, Travis should be stable again for rebased PRs.
> > Do we know when this is going to happen?
>
> PR is approved by Hynek and me, but travis didn't pass for the PR due
> the adapter tests exceed the 50 minutes limit. I am retrying one more
> time.
>
> I agree with what Bruno's suggested to split adapter tests into more
> groups. Or fix adapter tests to not require realm re-import after each
> test as that take significant amount of time. See [1]. But that will
> probably require some more changes and may be probably few days of work...
>
> [1]
>
>
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
>
> Marek
>
>
> >
> >> --Hynek
> >>
> >> [1] https://github.com/keycloak/keycloak/pull/6209
> >> [2] https://github.com/keycloak/keycloak/pull/6207
> >> [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> >> [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> >>
> >> On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org
> >> <mailto:bruno@abstractj.org>> wrote:
> >>
> >> Before answer this e-mail I was doing some experiments and there
> are
> >> some things that I identified we can do about this.
> >>
> >> 1. Enable cache again on Travis
> >>
> >> That won't fix the issue, but may help
> >>
> >> 2. Split the adapter tests
> >>
> >> I did some test with the following PR:
> >> https://github.com/keycloak/keycloak/pull/6221/files
> >>
> >> Here's the output:
> >> https://travis-ci.org/keycloak/keycloak/builds/567252894
> >>
> >> 3. Review the tests we run on Travis
> >>
> >> I'm not so sure if everything that we have running on Travis
> today is
> >> really required.
> >>
> >>
> >> For now if others agree on item 2, I can polish my PR and work to
> >> get it merged.
> >>
> >> On 2019-08-02, Stan Silvert wrote:
> >> > The adapter tests are timing out on all new PR's.
> >> >
> >> > Hynek, I'm told that you might be able to help?
> >> >
> >> > Stan
> >> >
> >> > _______________________________________________
> >> > keycloak-dev mailing list
> >> > keycloak-dev(a)lists.jboss.org <mailto:
> keycloak-dev(a)lists.jboss.org>
> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>
> >> --
> >>
> >> abstractj
> >>
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
>
7:46 a.m.
Thanks Hynek! I've approved the PR 6222 for run adapter authorization
tests conditionally. It seems this helped to have travis green again -
we will see if some more work is needed. My vote is to merge it ASAP.
Marek
On 06. 08. 19 10:39, Hynek Mlnarik wrote:
Requested Travis assistance as well [1]. There are now several
timeouts caused by compilation of keycloak taking too long, e.g. [2]
[1] https://travis-ci.community/t/travis-builds-fail-due-to-slowness/4571
[2] https://travis-ci.org/keycloak/keycloak/jobs/568226589
On Tue, Aug 6, 2019 at 9:56 AM Hynek Mlnarik <hmlnarik(a)redhat.com
<mailto:hmlnarik@redhat.com>> wrote:
Travis seems inconsistent re the times so better safe than sorry.
Hence I've reopened Bruno's PR [1] for splitting adapter tests,
but at the same time, I'd rather excluded authorization tests
completely since they take most of the time of adapter tests, and
only included those when appropriate [2].
However I'm not convinced that any of these PRs would help. I have
just cancelled a build stuck in the very early stage of the build
for almost 20 minutes (git clone, specifically "remote: Counting
objects: 100% (14222/14222), done." before build) which points to
infrastructure rather than testsuite issue.
--Hynek
[1] https://github.com/keycloak/keycloak/pull/6221
[2] https://github.com/keycloak/keycloak/pull/6222
On Mon, Aug 5, 2019 at 10:38 PM Marek Posolda <mposolda(a)redhat.com
<mailto:mposolda@redhat.com>> wrote:
On 05. 08. 19 19:13, Stan Silvert wrote:
> On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
>> The PRs are blocked by two issues:
>> 1) Adapter tests sometimes take too much time: This should be
>> addressed by (now merged) [1]
>> 2) OpenShiftTokenReviewEndpointTest being unstable,
addressed by [2]
>>
>> IIRC, we had issues with Travis caches [3, 4], but if there
is time it
>> might be worth checking whether the issues persist.
>>
>> Once [2] is merged, Travis should be stable again for
rebased PRs.
> Do we know when this is going to happen?
PR is approved by Hynek and me, but travis didn't pass for the
PR due
the adapter tests exceed the 50 minutes limit. I am retrying
one more time.
I agree with what Bruno's suggested to split adapter tests
into more
groups. Or fix adapter tests to not require realm re-import
after each
test as that take significant amount of time. See [1]. But
that will
probably require some more changes and may be probably few
days of work...
[1]
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
Marek
>
>> --Hynek
>>
>> [1] https://github.com/keycloak/keycloak/pull/6209
>> [2] https://github.com/keycloak/keycloak/pull/6207
>> [3] https://issues.jboss.org/browse/KEYCLOAK-5124
>> [4] https://issues.jboss.org/browse/KEYCLOAK-7285
>>
>> On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira
<bruno(a)abstractj.org <mailto:bruno@abstractj.org>
>> <mailto:bruno@abstractj.org
<mailto:bruno@abstractj.org>>>
wrote:
>>
>> Before answer this e-mail I was doing some experiments
and there are
>> some things that I identified we can do about this.
>>
>> 1. Enable cache again on Travis
>>
>> That won't fix the issue, but may help
>>
>> 2. Split the adapter tests
>>
>> I did some test with the following PR:
>> https://github.com/keycloak/keycloak/pull/6221/files
>>
>> Here's the output:
>> https://travis-ci.org/keycloak/keycloak/builds/567252894
>>
>> 3. Review the tests we run on Travis
>>
>> I'm not so sure if everything that we have running on
Travis today is
>> really required.
>>
>>
>> For now if others agree on item 2, I can polish my PR
and work to
>> get it merged.
>>
>> On 2019-08-02, Stan Silvert wrote:
>> > The adapter tests are timing out on all new PR's.
>> >
>> > Hynek, I'm told that you might be able to help?
>> >
>> > Stan
>> >
>> > _______________________________________________
>> > keycloak-dev mailing list
>> > keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
<mailto:keycloak-dev@lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>>
>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>> --
>>
>> abstractj
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
9:40 a.m.
I believe Hynek's changes help us a lot with the PRs and solves our
problem for now. Yesterday after merging Stan's PR which only touches
into non-Java files, the build from master started to fail for the
cross-dc Job.
Cross DC builds as Authz builds run conditionally against pull-request,
not against master. Here is my suggestion:
1. For these builds that have been consistently failing. We apply the
same conditions that we have for pull-request for master. Into other
words, if I didn't touch into cross-dc stuff, I don't run any tests
related to this when my changes are merged.
2. We have a Daily Job running on Travis against master. Into our
scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
"cron", we run all the tests: cross-dc, authz and others.
If that makes sense and we have some agreement about this. I can give it
a try.
On 2019-08-05, Marek Posolda wrote:
On 05. 08. 19 19:13, Stan Silvert wrote:
> On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> > The PRs are blocked by two issues:
> > 1) Adapter tests sometimes take too much time: This should be
> > addressed by (now merged) [1]
> > 2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
> >
> > IIRC, we had issues with Travis caches [3, 4], but if there is time it
> > might be worth checking whether the issues persist.
> >
> > Once [2] is merged, Travis should be stable again for rebased PRs.
> Do we know when this is going to happen?
PR is approved by Hynek and me, but travis didn't pass for the PR due the
adapter tests exceed the 50 minutes limit. I am retrying one more time.
I agree with what Bruno's suggested to split adapter tests into more groups.
Or fix adapter tests to not require realm re-import after each test as that
take significant amount of time. See [1]. But that will probably require
some more changes and may be probably few days of work...
[1]
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
Marek
>
> > --Hynek
> >
> > [1] https://github.com/keycloak/keycloak/pull/6209
> > [2] https://github.com/keycloak/keycloak/pull/6207
> > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> >
> > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org
> > <mailto:bruno@abstractj.org>> wrote:
> >
> > Before answer this e-mail I was doing some experiments and there are
> > some things that I identified we can do about this.
> >
> > 1. Enable cache again on Travis
> >
> > That won't fix the issue, but may help
> >
> > 2. Split the adapter tests
> >
> > I did some test with the following PR:
> > https://github.com/keycloak/keycloak/pull/6221/files
> >
> > Here's the output:
> > https://travis-ci.org/keycloak/keycloak/builds/567252894
> >
> > 3. Review the tests we run on Travis
> >
> > I'm not so sure if everything that we have running on Travis today is
> > really required.
> >
> >
> > For now if others agree on item 2, I can polish my PR and work to
> > get it merged.
> >
> > On 2019-08-02, Stan Silvert wrote:
> > > The adapter tests are timing out on all new PR's.
> > >
> > > Hynek, I'm told that you might be able to help?
> > >
> > > Stan
> > >
> > > _______________________________________________
> > > keycloak-dev mailing list
> > > keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> > --
> >
> > abstractj
> >
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
abstractj
2:45 a.m.
On Thu, Aug 8, 2019 at 5:32 PM Bruno Oliveira <bruno(a)abstractj.org> wrote:
I believe Hynek's changes help us a lot with the PRs and solves
our
problem for now. Yesterday after merging Stan's PR which only touches
into non-Java files, the build from master started to fail for the
cross-dc Job.
Cross DC builds as Authz builds run conditionally against pull-request,
not against master. Here is my suggestion:
1. For these builds that have been consistently failing. We apply the
same conditions that we have for pull-request for master. Into other
words, if I didn't touch into cross-dc stuff, I don't run any tests
related to this when my changes are merged.
Is there any benefit on implementing this conditional runs? Perhaps we
should always run the whole test suite. This way we could discover unstable
tests faster and hopefully get them fixed.
2. We have a Daily Job running on Travis against master. Into our
scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
"cron", we run all the tests: cross-dc, authz and others.
If that makes sense and we have some agreement about this. I can give
it
a try.
On 2019-08-05, Marek Posolda wrote:
> On 05. 08. 19 19:13, Stan Silvert wrote:
> > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> > > The PRs are blocked by two issues:
> > > 1) Adapter tests sometimes take too much time: This should be
> > > addressed by (now merged) [1]
> > > 2) OpenShiftTokenReviewEndpointTest being unstable, addressed by [2]
> > >
> > > IIRC, we had issues with Travis caches [3, 4], but if there is time
it
> > > might be worth checking whether the issues persist.
> > >
> > > Once [2] is merged, Travis should be stable again for rebased PRs.
> > Do we know when this is going to happen?
>
> PR is approved by Hynek and me, but travis didn't pass for the PR due the
> adapter tests exceed the 50 minutes limit. I am retrying one more time.
>
> I agree with what Bruno's suggested to split adapter tests into more
groups.
> Or fix adapter tests to not require realm re-import after each test as
that
> take significant amount of time. See [1]. But that will probably require
> some more changes and may be probably few days of work...
>
> [1]
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
>
> Marek
>
>
> >
> > > --Hynek
> > >
> > > [1] https://github.com/keycloak/keycloak/pull/6209
> > > [2] https://github.com/keycloak/keycloak/pull/6207
> > > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> > > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> > >
> > > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira <bruno(a)abstractj.org
> > > <mailto:bruno@abstractj.org>> wrote:
> > >
> > > Before answer this e-mail I was doing some experiments and
there are
> > > some things that I identified we can do about this.
> > >
> > > 1. Enable cache again on Travis
> > >
> > > That won't fix the issue, but may help
> > >
> > > 2. Split the adapter tests
> > >
> > > I did some test with the following PR:
> > > https://github.com/keycloak/keycloak/pull/6221/files
> > >
> > > Here's the output:
> > > https://travis-ci.org/keycloak/keycloak/builds/567252894
> > >
> > > 3. Review the tests we run on Travis
> > >
> > > I'm not so sure if everything that we have running on Travis
today is
> > > really required.
> > >
> > >
> > > For now if others agree on item 2, I can polish my PR and work
to
> > > get it merged.
> > >
> > > On 2019-08-02, Stan Silvert wrote:
> > > > The adapter tests are timing out on all new PR's.
> > > >
> > > > Hynek, I'm told that you might be able to help?
> > > >
> > > > Stan
> > > >
> > > > _______________________________________________
> > > > keycloak-dev mailing list
> > > > keycloak-dev(a)lists.jboss.org <mailto:
keycloak-dev(a)lists.jboss.org>
> > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > >
> > > --
> > >
> > > abstractj
> > >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
--
abstractj
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
3:19 a.m.
On 09. 08. 19 9:45, Sebastian Laskawiec wrote:
On Thu, Aug 8, 2019 at 5:32 PM Bruno Oliveira <bruno(a)abstractj.org
<mailto:bruno@abstractj.org>> wrote:
I believe Hynek's changes help us a lot with the PRs and solves our
problem for now. Yesterday after merging Stan's PR which only touches
into non-Java files, the build from master started to fail for the
cross-dc Job.
Cross DC builds as Authz builds run conditionally against
pull-request,
not against master. Here is my suggestion:
1. For these builds that have been consistently failing. We apply the
same conditions that we have for pull-request for master. Into other
words, if I didn't touch into cross-dc stuff, I don't run any tests
related to this when my changes are merged.
Is there any benefit on implementing this conditional runs? Perhaps we
should always run the whole test suite. This way we could discover
unstable tests faster and hopefully get them fixed.
For cross-dc tests, the problem is that they are not so stable. Running
it always will decrease travis stability and in most cases, running them
is not really necessarily. But as you pointed, there is possible issue
that we won't discover issues early and cross-dc tests can become broken
even with changes, which doesn't seem to be related to cross-dc (but
that happen only once during last 1.5 years since we have cross-dc
tests). So there are both pros and cons, but I vote to rather keep
cross-dc tests in travis conditional.
For authz tests, I am not sure. If they are stable, we can run them
always in theory. But I am not sure if they are really stable?
Marek
2. We have a Daily Job running on Travis against master. Into our
scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
"cron", we run all the tests: cross-dc, authz and others.
If that makes sense and we have some agreement about this. I can
give it
a try.
On 2019-08-05, Marek Posolda wrote:
> On 05. 08. 19 19:13, Stan Silvert wrote:
> > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> > > The PRs are blocked by two issues:
> > > 1) Adapter tests sometimes take too much time: This should be
> > > addressed by (now merged) [1]
> > > 2) OpenShiftTokenReviewEndpointTest being unstable,
addressed by [2]
> > >
> > > IIRC, we had issues with Travis caches [3, 4], but if there
is time it
> > > might be worth checking whether the issues persist.
> > >
> > > Once [2] is merged, Travis should be stable again for
rebased PRs.
> > Do we know when this is going to happen?
>
> PR is approved by Hynek and me, but travis didn't pass for the
PR due the
> adapter tests exceed the 50 minutes limit. I am retrying one
more time.
>
> I agree with what Bruno's suggested to split adapter tests into
more groups.
> Or fix adapter tests to not require realm re-import after each
test as that
> take significant amount of time. See [1]. But that will probably
require
> some more changes and may be probably few days of work...
>
> [1]
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
>
> Marek
>
>
> >
> > > --Hynek
> > >
> > > [1] https://github.com/keycloak/keycloak/pull/6209
> > > [2] https://github.com/keycloak/keycloak/pull/6207
> > > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> > > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> > >
> > > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira
<bruno(a)abstractj.org <mailto:bruno@abstractj.org>
> > > <mailto:bruno@abstractj.org
<mailto:bruno@abstractj.org>>>
wrote:
> > >
> > > Before answer this e-mail I was doing some experiments
and there are
> > > some things that I identified we can do about this.
> > >
> > > 1. Enable cache again on Travis
> > >
> > > That won't fix the issue, but may help
> > >
> > > 2. Split the adapter tests
> > >
> > > I did some test with the following PR:
> > > https://github.com/keycloak/keycloak/pull/6221/files
> > >
> > > Here's the output:
> > > https://travis-ci.org/keycloak/keycloak/builds/567252894
> > >
> > > 3. Review the tests we run on Travis
> > >
> > > I'm not so sure if everything that we have running on
Travis today is
> > > really required.
> > >
> > >
> > > For now if others agree on item 2, I can polish my PR
and work to
> > > get it merged.
> > >
> > > On 2019-08-02, Stan Silvert wrote:
> > > > The adapter tests are timing out on all new PR's.
> > > >
> > > > Hynek, I'm told that you might be able to help?
> > > >
> > > > Stan
> > > >
> > > > _______________________________________________
> > > > keycloak-dev mailing list
> > > > keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
<mailto:keycloak-dev@lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>>
> > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > >
> > > --
> > >
> > > abstractj
> > >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
--
abstractj
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
7:23 a.m.
On Fri, Aug 9, 2019 at 5:25 AM Marek Posolda <mposolda(a)redhat.com> wrote:
On 09. 08. 19 9:45, Sebastian Laskawiec wrote:
>
>
> On Thu, Aug 8, 2019 at 5:32 PM Bruno Oliveira <bruno(a)abstractj.org
> <mailto:bruno@abstractj.org>> wrote:
>
> I believe Hynek's changes help us a lot with the PRs and solves our
> problem for now. Yesterday after merging Stan's PR which only touches
> into non-Java files, the build from master started to fail for the
> cross-dc Job.
>
> Cross DC builds as Authz builds run conditionally against
> pull-request,
> not against master. Here is my suggestion:
>
> 1. For these builds that have been consistently failing. We apply the
> same conditions that we have for pull-request for master. Into other
> words, if I didn't touch into cross-dc stuff, I don't run any tests
> related to this when my changes are merged.
>
>
> Is there any benefit on implementing this conditional runs? Perhaps we
> should always run the whole test suite. This way we could discover
> unstable tests faster and hopefully get them fixed.
For cross-dc tests, the problem is that they are not so stable. Running
it always will decrease travis stability and in most cases, running them
is not really necessarily. But as you pointed, there is possible issue
that we won't discover issues early and cross-dc tests can become broken
even with changes, which doesn't seem to be related to cross-dc (but
that happen only once during last 1.5 years since we have cross-dc
tests). So there are both pros and cons, but I vote to rather keep
cross-dc tests in travis conditional.
For authz tests, I am not sure. If they are stable, we can run them
always in theory. But I am not sure if they are really stable?
They should be stable. In the adapter tests, the most critical one is using
angular, but after some changes from Michal (I think) the tests were
running just fine.
At least for my PRs, the errors I usually get are related with maven deps
not being fetched or cross-dc.
Marek
>
> 2. We have a Daily Job running on Travis against master. Into our
> scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
> "cron", we run all the tests: cross-dc, authz and others.
>
>
> If that makes sense and we have some agreement about this. I can
> give it
> a try.
>
> On 2019-08-05, Marek Posolda wrote:
> > On 05. 08. 19 19:13, Stan Silvert wrote:
> > > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> > > > The PRs are blocked by two issues:
> > > > 1) Adapter tests sometimes take too much time: This should be
> > > > addressed by (now merged) [1]
> > > > 2) OpenShiftTokenReviewEndpointTest being unstable,
> addressed by [2]
> > > >
> > > > IIRC, we had issues with Travis caches [3, 4], but if there
> is time it
> > > > might be worth checking whether the issues persist.
> > > >
> > > > Once [2] is merged, Travis should be stable again for
> rebased PRs.
> > > Do we know when this is going to happen?
> >
> > PR is approved by Hynek and me, but travis didn't pass for the
> PR due the
> > adapter tests exceed the 50 minutes limit. I am retrying one
> more time.
> >
> > I agree with what Bruno's suggested to split adapter tests into
> more groups.
> > Or fix adapter tests to not require realm re-import after each
> test as that
> > take significant amount of time. See [1]. But that will probably
> require
> > some more changes and may be probably few days of work...
> >
> > [1]
>
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
> >
> > Marek
> >
> >
> > >
> > > > --Hynek
> > > >
> > > > [1] https://github.com/keycloak/keycloak/pull/6209
> > > > [2] https://github.com/keycloak/keycloak/pull/6207
> > > > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> > > > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> > > >
> > > > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira
> <bruno(a)abstractj.org <mailto:bruno@abstractj.org>
> > > > <mailto:bruno@abstractj.org
<mailto:bruno@abstractj.org>>>
> wrote:
> > > >
> > > > Before answer this e-mail I was doing some experiments
> and there are
> > > > some things that I identified we can do about this.
> > > >
> > > > 1. Enable cache again on Travis
> > > >
> > > > That won't fix the issue, but may help
> > > >
> > > > 2. Split the adapter tests
> > > >
> > > > I did some test with the following PR:
> > > > https://github.com/keycloak/keycloak/pull/6221/files
> > > >
> > > > Here's the output:
> > > > https://travis-ci.org/keycloak/keycloak/builds/567252894
> > > >
> > > > 3. Review the tests we run on Travis
> > > >
> > > > I'm not so sure if everything that we have running on
> Travis today is
> > > > really required.
> > > >
> > > >
> > > > For now if others agree on item 2, I can polish my PR
> and work to
> > > > get it merged.
> > > >
> > > > On 2019-08-02, Stan Silvert wrote:
> > > > > The adapter tests are timing out on all new PR's.
> > > > >
> > > > > Hynek, I'm told that you might be able to help?
> > > > >
> > > > > Stan
> > > > >
> > > > > _______________________________________________
> > > > > keycloak-dev mailing list
> > > > > keycloak-dev(a)lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>
> <mailto:keycloak-dev@lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>>
> > > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > > >
> > > > --
> > > >
> > > > abstractj
> > > >
> > > _______________________________________________
> > > keycloak-dev mailing list
> > > keycloak-dev(a)lists.jboss.org <mailto:
keycloak-dev(a)lists.jboss.org>
> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> >
>
> --
>
> abstractj
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
7:34 a.m.
+1
It seems like things are back to normal on Travis
(https://www.traviscistatus.com/). Let's see how it goes.
On Fri, Aug 9, 2019 at 9:23 AM Pedro Igor Silva <psilva(a)redhat.com> wrote:
On Fri, Aug 9, 2019 at 5:25 AM Marek Posolda <mposolda(a)redhat.com> wrote:
>
> On 09. 08. 19 9:45, Sebastian Laskawiec wrote:
> >
> >
> > On Thu, Aug 8, 2019 at 5:32 PM Bruno Oliveira <bruno(a)abstractj.org
> > <mailto:bruno@abstractj.org>> wrote:
> >
> > I believe Hynek's changes help us a lot with the PRs and solves our
> > problem for now. Yesterday after merging Stan's PR which only touches
> > into non-Java files, the build from master started to fail for the
> > cross-dc Job.
> >
> > Cross DC builds as Authz builds run conditionally against
> > pull-request,
> > not against master. Here is my suggestion:
> >
> > 1. For these builds that have been consistently failing. We apply the
> > same conditions that we have for pull-request for master. Into other
> > words, if I didn't touch into cross-dc stuff, I don't run any tests
> > related to this when my changes are merged.
> >
> >
> > Is there any benefit on implementing this conditional runs? Perhaps we
> > should always run the whole test suite. This way we could discover
> > unstable tests faster and hopefully get them fixed.
>
> For cross-dc tests, the problem is that they are not so stable. Running
> it always will decrease travis stability and in most cases, running them
> is not really necessarily. But as you pointed, there is possible issue
> that we won't discover issues early and cross-dc tests can become broken
> even with changes, which doesn't seem to be related to cross-dc (but
> that happen only once during last 1.5 years since we have cross-dc
> tests). So there are both pros and cons, but I vote to rather keep
> cross-dc tests in travis conditional.
>
> For authz tests, I am not sure. If they are stable, we can run them
> always in theory. But I am not sure if they are really stable?
They should be stable. In the adapter tests, the most critical one is using angular, but
after some changes from Michal (I think) the tests were running just fine.
At least for my PRs, the errors I usually get are related with maven deps not being
fetched or cross-dc.
>
>
> Marek
>
> >
> > 2. We have a Daily Job running on Travis against master. Into our
> > scripts we identify the event using TRAVIS_EVENT_TYPE and if that's a
> > "cron", we run all the tests: cross-dc, authz and others.
> >
> >
> > If that makes sense and we have some agreement about this. I can
> > give it
> > a try.
> >
> > On 2019-08-05, Marek Posolda wrote:
> > > On 05. 08. 19 19:13, Stan Silvert wrote:
> > > > On 8/5/2019 3:43 AM, Hynek Mlnarik wrote:
> > > > > The PRs are blocked by two issues:
> > > > > 1) Adapter tests sometimes take too much time: This should
be
> > > > > addressed by (now merged) [1]
> > > > > 2) OpenShiftTokenReviewEndpointTest being unstable,
> > addressed by [2]
> > > > >
> > > > > IIRC, we had issues with Travis caches [3, 4], but if there
> > is time it
> > > > > might be worth checking whether the issues persist.
> > > > >
> > > > > Once [2] is merged, Travis should be stable again for
> > rebased PRs.
> > > > Do we know when this is going to happen?
> > >
> > > PR is approved by Hynek and me, but travis didn't pass for the
> > PR due the
> > > adapter tests exceed the 50 minutes limit. I am retrying one
> > more time.
> > >
> > > I agree with what Bruno's suggested to split adapter tests into
> > more groups.
> > > Or fix adapter tests to not require realm re-import after each
> > test as that
> > > take significant amount of time. See [1]. But that will probably
> > require
> > > some more changes and may be probably few days of work...
> > >
> > > [1]
> >
https://github.com/keycloak/keycloak/blob/master/testsuite/integration-ar...
> > >
> > > Marek
> > >
> > >
> > > >
> > > > > --Hynek
> > > > >
> > > > > [1] https://github.com/keycloak/keycloak/pull/6209
> > > > > [2] https://github.com/keycloak/keycloak/pull/6207
> > > > > [3] https://issues.jboss.org/browse/KEYCLOAK-5124
> > > > > [4] https://issues.jboss.org/browse/KEYCLOAK-7285
> > > > >
> > > > > On Sat, Aug 3, 2019 at 12:52 PM Bruno Oliveira
> > <bruno(a)abstractj.org <mailto:bruno@abstractj.org>
> > > > > <mailto:bruno@abstractj.org
<mailto:bruno@abstractj.org>>>
> > wrote:
> > > > >
> > > > > Before answer this e-mail I was doing some experiments
> > and there are
> > > > > some things that I identified we can do about this.
> > > > >
> > > > > 1. Enable cache again on Travis
> > > > >
> > > > > That won't fix the issue, but may help
> > > > >
> > > > > 2. Split the adapter tests
> > > > >
> > > > > I did some test with the following PR:
> > > > > https://github.com/keycloak/keycloak/pull/6221/files
> > > > >
> > > > > Here's the output:
> > > > > https://travis-ci.org/keycloak/keycloak/builds/567252894
> > > > >
> > > > > 3. Review the tests we run on Travis
> > > > >
> > > > > I'm not so sure if everything that we have running
on
> > Travis today is
> > > > > really required.
> > > > >
> > > > >
> > > > > For now if others agree on item 2, I can polish my PR
> > and work to
> > > > > get it merged.
> > > > >
> > > > > On 2019-08-02, Stan Silvert wrote:
> > > > > > The adapter tests are timing out on all new
PR's.
> > > > > >
> > > > > > Hynek, I'm told that you might be able to
help?
> > > > > >
> > > > > > Stan
> > > > > >
> > > > > > _______________________________________________
> > > > > > keycloak-dev mailing list
> > > > > > keycloak-dev(a)lists.jboss.org
> > <mailto:keycloak-dev@lists.jboss.org>
> > <mailto:keycloak-dev@lists.jboss.org
> > <mailto:keycloak-dev@lists.jboss.org>>
> > > > > >
https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > > > >
> > > > > --
> > > > >
> > > > > abstractj
> > > > >
> > > > _______________________________________________
> > > > keycloak-dev mailing list
> > > > keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
> > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > >
> > >
> >
> > --
> >
> > abstractj
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
- abstractj
1985
days inactive
1992
days old
14 comments
6 participants
participants (6)
-
Bruno Oliveira -
Hynek Mlnarik -
Marek Posolda -
Pedro Igor Silva -
Sebastian Laskawiec -
Stan Silvert