9:48 a.m.
Is there a good reason why travis does not run the tomcat and jetty
tests? I was just able to merge something that broke jetty.
Bill
1:02 p.m.
Is it a compilation error? I'm getting the same here.
On 2016-12-07, Bill Burke wrote:
Is there a good reason why travis does not run the tomcat and jetty
tests? I was just able to merge something that broke jetty.
Bill
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
abstractj
PGP: 0x84DC9914
3:43 p.m.
No, a test error. Working on my branch now.
On 12/7/16 1:02 PM, Bruno Oliveira wrote:
Is it a compilation error? I'm getting the same here.
On 2016-12-07, Bill Burke wrote:
> Is there a good reason why travis does not run the tomcat and jetty
> tests? I was just able to merge something that broke jetty.
>
> Bill
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
abstractj
PGP: 0x84DC9914
6:48 a.m.
They are not included in the script running travis
https://github.com/keycloak/keycloak/blob/master/travis-run-tests.sh#L4
It seems wee should add them here.
From the long run, it will be better to remove them (as they rely on
the old testsuite) and rather have them running on the
integration-arquillian testsuite. I know it sucks during development
that Jetty itself doesn't run in same JVM like the test (debugging etc).
But I hope we can have the solution, which will allow to choose whether
Jetty (or hopefully some other servers) are running in same JVM or
different JVM.
Marek
On 07/12/16 21:43, Bill Burke wrote:
No, a test error. Working on my branch now.
On 12/7/16 1:02 PM, Bruno Oliveira wrote:
> Is it a compilation error? I'm getting the same here.
>
> On 2016-12-07, Bill Burke wrote:
>> Is there a good reason why travis does not run the tomcat and jetty
>> tests? I was just able to merge something that broke jetty.
>>
>> Bill
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> --
>
> abstractj
> PGP: 0x84DC9914
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
9:05 a.m.
On 12/8/16 6:48 AM, Marek Posolda wrote:
They are not included in the script running travis
https://github.com/keycloak/keycloak/blob/master/travis-run-tests.sh#L4
It seems wee should add them here.
From the long run, it will be better to remove them (as they rely on
the old testsuite) and rather have them running on the
integration-arquillian testsuite. I know it sucks during development
that Jetty itself doesn't run in same JVM like the test (debugging
etc). But I hope we can have the solution, which will allow to choose
whether Jetty (or hopefully some other servers) are running in same
JVM or different JVM.
Unless arquillian can support running these adapter tests within an IDE,
they need to stay. In general, More often than not, I don't like the
arquillian style we've enforced because it takes a lot longer to write
tests, is often nearly impossible to do everything through a REST
interface, and is often really hard to debug. Maybe its just the types
of things I'm testing. Some of the tests should not have been ported to
arquillian, or, at least shouldn't have been ported to using a REST
interface for the entire test.
Bill
11:47 a.m.
Arquillian can support that. All we need is just different
implementations of arquillian DeployableContainer interface to represent
various containers. I've managed to have adapter tests already running
on embedded undertow and everything is running in single JVM.
Development and debugging is fine with that.
Having the test+server+adapter in single JVM just rocks during
development. Problem is, that it won't handle all type of errors. For
example if Jetty adapter ZIP distribution is broken (eg. some JAR file
is missing in it), the Jetty adapter tests, which uses embedded Jetty
won't catch that. For those reasons, also QA team needs to test with
"real" containers, not with the embedded ones.
On 08/12/16 15:05, Bill Burke wrote:
On 12/8/16 6:48 AM, Marek Posolda wrote:
> They are not included in the script running travis
> https://github.com/keycloak/keycloak/blob/master/travis-run-tests.sh#L4
>
> It seems wee should add them here.
>
> From the long run, it will be better to remove them (as they rely on
> the old testsuite) and rather have them running on the
> integration-arquillian testsuite. I know it sucks during development
> that Jetty itself doesn't run in same JVM like the test (debugging
> etc). But I hope we can have the solution, which will allow to choose
> whether Jetty (or hopefully some other servers) are running in same
> JVM or different JVM.
>
Unless arquillian can support running these adapter tests within an
IDE, they need to stay.
Arquillian can support that. All we need is just different
implementations of arquillian DeployableContainer interface to represent
various containers. I've managed to have adapter tests already running
on embedded undertow and everything is running in single JVM.
Development and debugging is fine with that.
In general, More often than not, I don't like the arquillian
style
we've enforced because it takes a lot longer to write tests, is often
nearly impossible to do everything through a REST interface, and is
often really hard to debug. Maybe its just the types of things I'm
testing. Some of the tests should not have been ported to arquillian,
or, at least shouldn't have been ported to using a REST interface for
the entire test.
Yeah, having the test+server+adapter in single JVM just rocks
during
development. Problem is, that it won't handle all type of errors. For
example if Jetty adapter ZIP distribution is broken (eg. some JAR file
is missing), the Jetty adapter tests, which uses embedded Jetty, won't
catch that. For those and similar reasons, also QA team needs to test
with "real" containers, not with the embedded ones.
IMO ideal is, if we can have both possibilities. Test with embedded
container during development, but also has possibility to run the same
test against external container in different JVM. Arquillian testsuite
can support both approaches, but the old testsuite doesn't support the
external...
I agree some tests shouldn't use REST endpoints. Like the model tests
from the old testsuite. But they still need to be able to run on
external environment though. I guess we can have something like the
"bridge" test, which will just deploy the model test to the REST
endpoint and you will be able to test things inside this REST endpoint
on server side? So you will be able to use all the stuff like model
classes etc.
Marek
Bill
11:50 a.m.
IMO ideal is, if we can have both possibilities. Test with embedded
container during development, but also has possibility to run the same
test against external container in different JVM. Arquillian testsuite
can support both approaches, but the old testsuite doesn't support the
external...
I agree some tests shouldn't use REST endpoints. Like the model tests
from the old testsuite. But they still need to be able to run on
external environment though. I guess we can have something like the
"bridge" test, which will just deploy the model test to the REST
endpoint and you will be able to test things inside this REST endpoint
on server side? So you will be able to use all the stuff like model
classes etc.
Marek
On 08/12/16 17:47, Marek Posolda wrote:
Arquillian can support that. All we need is just different
implementations of arquillian DeployableContainer interface to
represent various containers. I've managed to have adapter tests
already running on embedded undertow and everything is running in
single JVM. Development and debugging is fine with that.
Having the test+server+adapter in single JVM just rocks during
development. Problem is, that it won't handle all type of errors. For
example if Jetty adapter ZIP distribution is broken (eg. some JAR file
is missing in it), the Jetty adapter tests, which uses embedded Jetty
won't catch that. For those reasons, also QA team needs to test with
"real" containers, not with the embedded ones.
On 08/12/16 15:05, Bill Burke wrote:
>
>
> On 12/8/16 6:48 AM, Marek Posolda wrote:
>> They are not included in the script running travis
>> https://github.com/keycloak/keycloak/blob/master/travis-run-tests.sh#L4
>>
>> It seems wee should add them here.
>>
>> From the long run, it will be better to remove them (as they rely on
>> the old testsuite) and rather have them running on the
>> integration-arquillian testsuite. I know it sucks during development
>> that Jetty itself doesn't run in same JVM like the test (debugging
>> etc). But I hope we can have the solution, which will allow to
>> choose whether Jetty (or hopefully some other servers) are running
>> in same JVM or different JVM.
>>
> Unless arquillian can support running these adapter tests within an
> IDE, they need to stay.
Arquillian can support that. All we need is just different
implementations of arquillian DeployableContainer interface to
represent various containers. I've managed to have adapter tests
already running on embedded undertow and everything is running in
single JVM. Development and debugging is fine with that.
> In general, More often than not, I don't like the arquillian style
> we've enforced because it takes a lot longer to write tests, is often
> nearly impossible to do everything through a REST interface, and is
> often really hard to debug. Maybe its just the types of things I'm
> testing. Some of the tests should not have been ported to
> arquillian, or, at least shouldn't have been ported to using a REST
> interface for the entire test.
Yeah, having the test+server+adapter in single JVM just rocks during
development. Problem is, that it won't handle all type of errors. For
example if Jetty adapter ZIP distribution is broken (eg. some JAR file
is missing), the Jetty adapter tests, which uses embedded Jetty, won't
catch that. For those and similar reasons, also QA team needs to test
with "real" containers, not with the embedded ones.
IMO ideal is, if we can have both possibilities. Test with embedded
container during development, but also has possibility to run the same
test against external container in different JVM. Arquillian testsuite
can support both approaches, but the old testsuite doesn't support the
external...
I agree some tests shouldn't use REST endpoints. Like the model tests
from the old testsuite. But they still need to be able to run on
external environment though. I guess we can have something like the
"bridge" test, which will just deploy the model test to the REST
endpoint and you will be able to test things inside this REST endpoint
on server side? So you will be able to use all the stuff like model
classes etc.
Marek
>
> Bill
2:28 a.m.
Ahoj Marku,
On 12/08/2016 05:50 PM, Marek Posolda wrote:
IMO ideal is, if we can have both possibilities. Test with embedded
container during development, but also has possibility to run the same
test against external container in different JVM. Arquillian testsuite
can support both approaches, but the old testsuite doesn't support the
external...
I agree some tests shouldn't use REST endpoints. Like the model tests
from the old testsuite. But they still need to be able to run on
external environment though. I guess we can have something like the
"bridge" test, which will just deploy the model test to the REST
endpoint and you will be able to test things inside this REST endpoint
on server side? So you will be able to use all the stuff like model
classes etc.
There is https://issues.jboss.org/browse/KEYCLOAK-3729 it seems to me
that your proposal with "bridge" test could resolve this. It would be
great to have a meeting to discuss details or if you will have time to
make PoC. I'd love to take a look at it.
Marek
On 08/12/16 17:47, Marek Posolda wrote:
> Arquillian can support that. All we need is just different
> implementations of arquillian DeployableContainer interface to
> represent various containers. I've managed to have adapter tests
> already running on embedded undertow and everything is running in
> single JVM. Development and debugging is fine with that.
>
> Having the test+server+adapter in single JVM just rocks during
> development. Problem is, that it won't handle all type of errors. For
> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
> is missing in it), the Jetty adapter tests, which uses embedded Jetty
> won't catch that. For those reasons, also QA team needs to test with
> "real" containers, not with the embedded ones.
>
>
>
> On 08/12/16 15:05, Bill Burke wrote:
>>
>> On 12/8/16 6:48 AM, Marek Posolda wrote:
>>> They are not included in the script running travis
>>> https://github.com/keycloak/keycloak/blob/master/travis-run-tests.sh#L4
>>>
>>> It seems wee should add them here.
>>>
>>> From the long run, it will be better to remove them (as they rely on
>>> the old testsuite) and rather have them running on the
>>> integration-arquillian testsuite. I know it sucks during development
>>> that Jetty itself doesn't run in same JVM like the test (debugging
>>> etc). But I hope we can have the solution, which will allow to
>>> choose whether Jetty (or hopefully some other servers) are running
>>> in same JVM or different JVM.
>>>
>> Unless arquillian can support running these adapter tests within an
>> IDE, they need to stay.
> Arquillian can support that. All we need is just different
> implementations of arquillian DeployableContainer interface to
> represent various containers. I've managed to have adapter tests
> already running on embedded undertow and everything is running in
> single JVM. Development and debugging is fine with that.
>> In general, More often than not, I don't like the arquillian style
>> we've enforced because it takes a lot longer to write tests, is often
>> nearly impossible to do everything through a REST interface, and is
>> often really hard to debug. Maybe its just the types of things I'm
>> testing. Some of the tests should not have been ported to
>> arquillian, or, at least shouldn't have been ported to using a REST
>> interface for the entire test.
> Yeah, having the test+server+adapter in single JVM just rocks during
> development. Problem is, that it won't handle all type of errors. For
> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
> is missing), the Jetty adapter tests, which uses embedded Jetty, won't
> catch that. For those and similar reasons, also QA team needs to test
> with "real" containers, not with the embedded ones.
>
> IMO ideal is, if we can have both possibilities. Test with embedded
> container during development, but also has possibility to run the same
> test against external container in different JVM. Arquillian testsuite
> can support both approaches, but the old testsuite doesn't support the
> external...
>
> I agree some tests shouldn't use REST endpoints. Like the model tests
> from the old testsuite. But they still need to be able to run on
> external environment though. I guess we can have something like the
> "bridge" test, which will just deploy the model test to the REST
> endpoint and you will be able to test things inside this REST endpoint
> on server side? So you will be able to use all the stuff like model
> classes etc.
>
> Marek
>> Bill
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
2:51 a.m.
Ahoj Vlasto,
On 09/12/16 08:28, Vlasta Ramik wrote:
Ahoj Marku,
On 12/08/2016 05:50 PM, Marek Posolda wrote:
> IMO ideal is, if we can have both possibilities. Test with embedded
> container during development, but also has possibility to run the same
> test against external container in different JVM. Arquillian testsuite
> can support both approaches, but the old testsuite doesn't support the
> external...
>
> I agree some tests shouldn't use REST endpoints. Like the model tests
> from the old testsuite. But they still need to be able to run on
> external environment though. I guess we can have something like the
> "bridge" test, which will just deploy the model test to the REST
> endpoint and you will be able to test things inside this REST endpoint
> on server side? So you will be able to use all the stuff like model
> classes etc.
There is https://issues.jboss.org/browse/KEYCLOAK-3729 it seems to me
that your proposal with "bridge" test could resolve this. It would be
great to have a meeting to discuss details or if you will have time to
make PoC. I'd love to take a look at it.
ah, so you already tried different
approach with the expose
KeycloakSessionFactory to JNDI and remote lookup it from tests. This
seems much easier then my approach based on deploying test on REST
endpoint. But for the JNDI, I have no clue about the things like
serialization etc. Maybe there is some easy solution to it, but I don't
know ATM... I am sure that approach based on REST endpoints will work,
but it's more work.
+1 for the meeting. Maybe next week or after Christmas? I personally
won't have time to do any PoC or something around this before Christmas :(
Marek
> Marek
>
> On 08/12/16 17:47, Marek Posolda wrote:
>> Arquillian can support that. All we need is just different
>> implementations of arquillian DeployableContainer interface to
>> represent various containers. I've managed to have adapter tests
>> already running on embedded undertow and everything is running in
>> single JVM. Development and debugging is fine with that.
>>
>> Having the test+server+adapter in single JVM just rocks during
>> development. Problem is, that it won't handle all type of errors. For
>> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
>> is missing in it), the Jetty adapter tests, which uses embedded Jetty
>> won't catch that. For those reasons, also QA team needs to test with
>> "real" containers, not with the embedded ones.
>>
>>
>>
>> On 08/12/16 15:05, Bill Burke wrote:
>>> On 12/8/16 6:48 AM, Marek Posolda wrote:
>>>> They are not included in the script running travis
>>>> https://github.com/keycloak/keycloak/blob/master/travis-run-tests.sh#L4
>>>>
>>>> It seems wee should add them here.
>>>>
>>>> From the long run, it will be better to remove them (as they rely on
>>>> the old testsuite) and rather have them running on the
>>>> integration-arquillian testsuite. I know it sucks during development
>>>> that Jetty itself doesn't run in same JVM like the test (debugging
>>>> etc). But I hope we can have the solution, which will allow to
>>>> choose whether Jetty (or hopefully some other servers) are running
>>>> in same JVM or different JVM.
>>>>
>>> Unless arquillian can support running these adapter tests within an
>>> IDE, they need to stay.
>> Arquillian can support that. All we need is just different
>> implementations of arquillian DeployableContainer interface to
>> represent various containers. I've managed to have adapter tests
>> already running on embedded undertow and everything is running in
>> single JVM. Development and debugging is fine with that.
>>> In general, More often than not, I don't like the arquillian style
>>> we've enforced because it takes a lot longer to write tests, is often
>>> nearly impossible to do everything through a REST interface, and is
>>> often really hard to debug. Maybe its just the types of things I'm
>>> testing. Some of the tests should not have been ported to
>>> arquillian, or, at least shouldn't have been ported to using a REST
>>> interface for the entire test.
>> Yeah, having the test+server+adapter in single JVM just rocks during
>> development. Problem is, that it won't handle all type of errors. For
>> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
>> is missing), the Jetty adapter tests, which uses embedded Jetty, won't
>> catch that. For those and similar reasons, also QA team needs to test
>> with "real" containers, not with the embedded ones.
>>
>> IMO ideal is, if we can have both possibilities. Test with embedded
>> container during development, but also has possibility to run the same
>> test against external container in different JVM. Arquillian testsuite
>> can support both approaches, but the old testsuite doesn't support the
>> external...
>>
>> I agree some tests shouldn't use REST endpoints. Like the model tests
>> from the old testsuite. But they still need to be able to run on
>> external environment though. I guess we can have something like the
>> "bridge" test, which will just deploy the model test to the REST
>> endpoint and you will be able to test things inside this REST endpoint
>> on server side? So you will be able to use all the stuff like model
>> classes etc.
>>
>> Marek
>>> Bill
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
8:19 a.m.
Let's put this on hold for after 7.1 GA and make it a priority after that.
We are going to use Arquillian bases tests and removing the old tests as
they have less value from a QE perspective. This has been discussed many
times now and we have agreed that this will be done a long time ago so
please let's not reiterate this discussion again and again. If dev and QE
are going to collaborate on the tests we need to share the approach and QE
has to have full functional tests.
We need to resolve this issue though so we can get access to the
KeycloakSession from within certain tests. Ability to run adapter tests
from within the IDE is also a nice to have, but that may be harder to
achieve (and the effort may not be worth it, at least not for all adapters).
On 9 December 2016 at 08:51, Marek Posolda <mposolda(a)redhat.com> wrote:
Ahoj Vlasto,
On 09/12/16 08:28, Vlasta Ramik wrote:
>
> Ahoj Marku,
>
>
> On 12/08/2016 05:50 PM, Marek Posolda wrote:
>> IMO ideal is, if we can have both possibilities. Test with embedded
>> container during development, but also has possibility to run the same
>> test against external container in different JVM. Arquillian testsuite
>> can support both approaches, but the old testsuite doesn't support the
>> external...
>>
>> I agree some tests shouldn't use REST endpoints. Like the model tests
>> from the old testsuite. But they still need to be able to run on
>> external environment though. I guess we can have something like the
>> "bridge" test, which will just deploy the model test to the REST
>> endpoint and you will be able to test things inside this REST endpoint
>> on server side? So you will be able to use all the stuff like model
>> classes etc.
> There is https://issues.jboss.org/browse/KEYCLOAK-3729 it seems to me
> that your proposal with "bridge" test could resolve this. It would be
> great to have a meeting to discuss details or if you will have time to
> make PoC. I'd love to take a look at it.
ah, so you already tried different approach with the expose
KeycloakSessionFactory to JNDI and remote lookup it from tests. This
seems much easier then my approach based on deploying test on REST
endpoint. But for the JNDI, I have no clue about the things like
serialization etc. Maybe there is some easy solution to it, but I don't
know ATM... I am sure that approach based on REST endpoints will work,
but it's more work.
+1 for the meeting. Maybe next week or after Christmas? I personally
won't have time to do any PoC or something around this before Christmas :(
Marek
>> Marek
>>
>> On 08/12/16 17:47, Marek Posolda wrote:
>>> Arquillian can support that. All we need is just different
>>> implementations of arquillian DeployableContainer interface to
>>> represent various containers. I've managed to have adapter tests
>>> already running on embedded undertow and everything is running in
>>> single JVM. Development and debugging is fine with that.
>>>
>>> Having the test+server+adapter in single JVM just rocks during
>>> development. Problem is, that it won't handle all type of errors. For
>>> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
>>> is missing in it), the Jetty adapter tests, which uses embedded Jetty
>>> won't catch that. For those reasons, also QA team needs to test with
>>> "real" containers, not with the embedded ones.
>>>
>>>
>>>
>>> On 08/12/16 15:05, Bill Burke wrote:
>>>> On 12/8/16 6:48 AM, Marek Posolda wrote:
>>>>> They are not included in the script running travis
>>>>> https://github.com/keycloak/keycloak/blob/master/travis-
run-tests.sh#L4
>>>>>
>>>>> It seems wee should add them here.
>>>>>
>>>>> From the long run, it will be better to remove them (as they rely
on
>>>>> the old testsuite) and rather have them running on the
>>>>> integration-arquillian testsuite. I know it sucks during
development
>>>>> that Jetty itself doesn't run in same JVM like the test
(debugging
>>>>> etc). But I hope we can have the solution, which will allow to
>>>>> choose whether Jetty (or hopefully some other servers) are running
>>>>> in same JVM or different JVM.
>>>>>
>>>> Unless arquillian can support running these adapter tests within an
>>>> IDE, they need to stay.
>>> Arquillian can support that. All we need is just different
>>> implementations of arquillian DeployableContainer interface to
>>> represent various containers. I've managed to have adapter tests
>>> already running on embedded undertow and everything is running in
>>> single JVM. Development and debugging is fine with that.
>>>> In general, More often than not, I don't like the arquillian style
>>>> we've enforced because it takes a lot longer to write tests, is
often
>>>> nearly impossible to do everything through a REST interface, and is
>>>> often really hard to debug. Maybe its just the types of things I'm
>>>> testing. Some of the tests should not have been ported to
>>>> arquillian, or, at least shouldn't have been ported to using a REST
>>>> interface for the entire test.
>>> Yeah, having the test+server+adapter in single JVM just rocks during
>>> development. Problem is, that it won't handle all type of errors. For
>>> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
>>> is missing), the Jetty adapter tests, which uses embedded Jetty, won't
>>> catch that. For those and similar reasons, also QA team needs to test
>>> with "real" containers, not with the embedded ones.
>>>
>>> IMO ideal is, if we can have both possibilities. Test with embedded
>>> container during development, but also has possibility to run the same
>>> test against external container in different JVM. Arquillian testsuite
>>> can support both approaches, but the old testsuite doesn't support the
>>> external...
>>>
>>> I agree some tests shouldn't use REST endpoints. Like the model tests
>>> from the old testsuite. But they still need to be able to run on
>>> external environment though. I guess we can have something like the
>>> "bridge" test, which will just deploy the model test to the REST
>>> endpoint and you will be able to test things inside this REST endpoint
>>> on server side? So you will be able to use all the stuff like model
>>> classes etc.
>>>
>>> Marek
>>>> Bill
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
10:22 a.m.
On 12/12/16 8:19 AM, Stian Thorgersen wrote:
Let's put this on hold for after 7.1 GA and make it a priority
after that.
We can't have tomcat and jetty adapters failing and regressions
happening. There's a lot of people that depend on them. Our customer
base comes from a successful community project. You don't start
undercutting community just because its not supported in product. I'm
adding them back to the build because its the right thing to do. I
shouldn't have even asked and just did it, but I just found it quite
annoying that they were removed as I knew they were removed on purpose.
We are going to use Arquillian bases tests and removing the old tests
as
they have less value from a QE perspective.
Negative. You are just 100% wrong on
this. I've said this over and
over and you just don't listen and its getting really annoying. There
are many tests which it doesn't matter if they run in the container or
not. Many tests it is not possible to even run as its impossible to set
up the conditions you want to happen. Many tests are functional SPI
tests and it doesn't matter at all if they run inside the app server or
not and its much easier to setup and debug.
This has been discussed many
times now and we have agreed that this will be done a long time ago so
please let's not reiterate this discussion again and again. If dev and QE
are going to collaborate on the tests we need to share the approach and QE
has to have full functional tests.
Discussed many times and I never agreed to
anything. And kept telling
you over and over that this mandate is uneccessary, hurtful to
development, and is a lot of wasted effort, but you don't want to listen.
We need to resolve this issue though so we can get access to the
KeycloakSession from within certain tests. Ability to run adapter tests
from within the IDE is also a nice to have, but that may be harder to
achieve (and the effort may not be worth it, at least not for all adapters).
There's a difference between unit tests, functional tests, and
integration tests that you are missing. Everything is not an
integration test. Its unacceptable for adapter tests to not be able to
run in the IDE. If you want to have additional non-IDE runnable
integration tests, that is fine, but to be able to develop new features
and debug most problems, running in the IDE is a must and you're just
being stubborn to state otherwise. Its quite annoying.
Bill
11:31 a.m.
On 12 December 2016 at 16:22, Bill Burke <bburke(a)redhat.com> wrote:
On 12/12/16 8:19 AM, Stian Thorgersen wrote:
> Let's put this on hold for after 7.1 GA and make it a priority after
that.
We can't have tomcat and jetty adapters failing and regressions
happening. There's a lot of people that depend on them. Our customer
base comes from a successful community project. You don't start
undercutting community just because its not supported in product. I'm
adding them back to the build because its the right thing to do. I
shouldn't have even asked and just did it, but I just found it quite
annoying that they were removed as I knew they were removed on purpose.
You completely miss understood me. No one removed them on purpose. AFAIK
these tests have never actually ran on Travis.
No one has ever said that we are undercutting the community and that Tomcat
and Jetty is not still important. I've got absolutely no clue why you think
anyone has suggested that?!
> We are going to use Arquillian bases tests and removing the old tests as
> they have less value from a QE perspective.
Negative. You are just 100% wrong on this. I've said this over and
over and you just don't listen and its getting really annoying. There
are many tests which it doesn't matter if they run in the container or
not. Many tests it is not possible to even run as its impossible to set
up the conditions you want to happen. Many tests are functional SPI
tests and it doesn't matter at all if they run inside the app server or
not and its much easier to setup and debug
All tests matter if they run in the container! Bad imports, wrong module
definitons and bad/different versions of dependencies can all cause tests
to pass in the embedded/fake container while not working properly when
running within WildFly/EAP.
All tests that rely on the DB layer also needs to test different DB
vendors! Setting up and configuring CI is already complicated for the real
containers and that effort would have to be duplicated for a fake
environment as datasources are not available.
.
> This has been discussed many
> times now and we have agreed that this will be done a long time ago so
> please let's not reiterate this discussion again and again. If dev and QE
> are going to collaborate on the tests we need to share the approach and
QE
> has to have full functional tests.
Discussed many times and I never agreed to anything. And kept telling
you over and over that this mandate is uneccessary, hurtful to
development, and is a lot of wasted effort, but you don't want to listen.
See above
> We need to resolve this issue though so we can get access to the
> KeycloakSession from within certain tests. Ability to run adapter tests
> from within the IDE is also a nice to have, but that may be harder to
> achieve (and the effort may not be worth it, at least not for all
adapters).
There's a difference between unit tests, functional tests, and
integration tests that you are missing. Everything is not an
integration test. Its unacceptable for adapter tests to not be able to
run in the IDE. If you want to have additional non-IDE runnable
integration tests, that is fine, but to be able to develop new features
and debug most problems, running in the IDE is a must and you're just
being stubborn to state otherwise. Its quite annoying.
I agree that adapter tests should run within the IDE, but I disagree with
the approach you are suggesting. Adapter tests are even more important to
have a real environment IMO.
Bill
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
11:37 a.m.
To add - I've said many many times that we need to be able to get the same
access to the server as we have from the old testsuite and get direct
access to KeycloakSession from within tests. So your argument about tests
not being possi8ble to run is bs.
On 12 December 2016 at 17:31, Stian Thorgersen <sthorger(a)redhat.com> wrote:
On 12 December 2016 at 16:22, Bill Burke <bburke(a)redhat.com> wrote:
>
>
> On 12/12/16 8:19 AM, Stian Thorgersen wrote:
> > Let's put this on hold for after 7.1 GA and make it a priority after
> that.
> We can't have tomcat and jetty adapters failing and regressions
> happening. There's a lot of people that depend on them. Our customer
> base comes from a successful community project. You don't start
> undercutting community just because its not supported in product. I'm
> adding them back to the build because its the right thing to do. I
> shouldn't have even asked and just did it, but I just found it quite
> annoying that they were removed as I knew they were removed on purpose.
>
You completely miss understood me. No one removed them on purpose. AFAIK
these tests have never actually ran on Travis.
No one has ever said that we are undercutting the community and that
Tomcat and Jetty is not still important. I've got absolutely no clue why
you think anyone has suggested that?!
>
> > We are going to use Arquillian bases tests and removing the old tests as
> > they have less value from a QE perspective.
> Negative. You are just 100% wrong on this. I've said this over and
> over and you just don't listen and its getting really annoying. There
> are many tests which it doesn't matter if they run in the container or
> not. Many tests it is not possible to even run as its impossible to set
> up the conditions you want to happen. Many tests are functional SPI
> tests and it doesn't matter at all if they run inside the app server or
> not and its much easier to setup and debug
All tests matter if they run in the container! Bad imports, wrong module
definitons and bad/different versions of dependencies can all cause tests
to pass in the embedded/fake container while not working properly when
running within WildFly/EAP.
All tests that rely on the DB layer also needs to test different DB
vendors! Setting up and configuring CI is already complicated for the real
containers and that effort would have to be duplicated for a fake
environment as datasources are not available.
> .
>
>
> > This has been discussed many
> > times now and we have agreed that this will be done a long time ago so
> > please let's not reiterate this discussion again and again. If dev and
> QE
> > are going to collaborate on the tests we need to share the approach and
> QE
> > has to have full functional tests.
> Discussed many times and I never agreed to anything. And kept telling
> you over and over that this mandate is uneccessary, hurtful to
> development, and is a lot of wasted effort, but you don't want to listen.
>
See above
>
> > We need to resolve this issue though so we can get access to the
> > KeycloakSession from within certain tests. Ability to run adapter tests
> > from within the IDE is also a nice to have, but that may be harder to
> > achieve (and the effort may not be worth it, at least not for all
> adapters).
> There's a difference between unit tests, functional tests, and
> integration tests that you are missing. Everything is not an
> integration test. Its unacceptable for adapter tests to not be able to
> run in the IDE. If you want to have additional non-IDE runnable
> integration tests, that is fine, but to be able to develop new features
> and debug most problems, running in the IDE is a must and you're just
> being stubborn to state otherwise. Its quite annoying.
>
I agree that adapter tests should run within the IDE, but I disagree
with the approach you are suggesting. Adapter tests are even more important
to have a real environment IMO.
>
> Bill
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
3:17 a.m.
Dne 12.12.2016 v 16:22 Bill Burke napsal(a):
On 12/12/16 8:19 AM, Stian Thorgersen wrote:
> Let's put this on hold for after 7.1 GA and make it a priority after that.
We can't have tomcat and jetty adapters failing and regressions
happening. There's a lot of people that depend on them. Our customer
base comes from a successful community project. You don't start
undercutting community just because its not supported in product. I'm
adding them back to the build because its the right thing to do. I
shouldn't have even asked and just did it, but I just found it quite
annoying that they were removed as I knew they were removed on purpose.
> We are going to use Arquillian bases tests and removing the old tests as
> they have less value from a QE perspective.
Negative. You are just 100% wrong on this. I've said this over and
over and you just don't listen and its getting really annoying. There
are many tests which it doesn't matter if they run in the container or
not. Many tests it is not possible to even run as its impossible to set
up the conditions you want to happen. Many tests are functional SPI
tests and it doesn't matter at all if they run inside the app server or
not and its much easier to setup and debug.
Running the tests in the container is crucial requirement for QE. We
have to run the tests against the container built by productization,
there is no other option here. We should be able to test RH-SSO/Keycloak
patches, RH-SSO/Keycloak with EAP CPs. There can be many errors which we
are not able to catch in embedded container.
If we will not able to collaborate on the tests and will not use the
shared approach, it will be a big step back in our effort to correctly
setup CI jobs for community and the product.
> This has been discussed many
> times now and we have agreed that this will be done a long time ago so
> please let's not reiterate this discussion again and again. If dev and QE
> are going to collaborate on the tests we need to share the approach and QE
> has to have full functional tests.
Discussed many times and I never agreed to anything. And kept telling
you over and over that this mandate is uneccessary, hurtful to
development, and is a lot of wasted effort, but you don't want to listen.
> We need to resolve this issue though so we can get access to the
> KeycloakSession from within certain tests. Ability to run adapter tests
> from within the IDE is also a nice to have, but that may be harder to
> achieve (and the effort may not be worth it, at least not for all adapters).
There's a difference between unit tests, functional tests, and
integration tests that you are missing. Everything is not an
integration test. Its unacceptable for adapter tests to not be able to
run in the IDE. If you want to have additional non-IDE runnable
integration tests, that is fine, but to be able to develop new features
and debug most problems, running in the IDE is a must and you're just
being stubborn to state otherwise. Its quite annoying.
Bill
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
2689
days inactive
2695
days old
13 comments
6 participants
participants (6)
-
Bill Burke -
Bruno Oliveira -
Marek Posolda -
Pavel Drozd -
Stian Thorgersen -
Vlasta Ramik