[keycloak-dev] Why aren't tomcat and jetty tests run!?

[Vlasta Ramik]

Ahoj Marku,


On 12/08/2016 05:50 PM, Marek Posolda wrote:
> IMO ideal is, if we can have both possibilities. Test with embedded
> container during development, but also has possibility to run the same
> test against external container in different JVM. Arquillian testsuite
> can support both approaches, but the old testsuite doesn't support the
> external...
>
> I agree some tests shouldn't use REST endpoints. Like the model tests
> from the old testsuite. But they still need to be able to run on
> external environment though. I guess we can have something like the
> "bridge" test, which will just deploy the model test to the REST
> endpoint and you will be able to test things inside this REST endpoint
> on server side? So you will be able to use all the stuff like model
> classes etc.
There is https://issues.jboss.org/browse/KEYCLOAK-3729 it seems to me 
that your proposal with "bridge" test could resolve this. It would be 
great to have a meeting to discuss details or if you will have time to 
make PoC. I'd love to take a look at it.
>
> Marek
>
> On 08/12/16 17:47, Marek Posolda wrote:
>> Arquillian can support that. All we need is just different
>> implementations of arquillian DeployableContainer interface to
>> represent various containers. I've managed to have adapter tests
>> already running on embedded undertow and everything is running in
>> single JVM. Development and debugging is fine with that.
>>
>> Having the test+server+adapter in single JVM just rocks during
>> development. Problem is, that it won't handle all type of errors. For
>> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
>> is missing in it), the Jetty adapter tests, which uses embedded Jetty
>> won't catch that. For those reasons, also QA team needs to test with
>> "real" containers, not with the embedded ones.
>>
>>
>>
>> On 08/12/16 15:05, Bill Burke wrote:
>>>
>>> On 12/8/16 6:48 AM, Marek Posolda wrote:
>>>> They are not included in the script running travis
>>>> https://github.com/keycloak/keycloak/blob/master/travis-run-tests.sh#L4
>>>>
>>>> It seems wee should add them here.
>>>>
>>>>  From the long run, it will be better to remove them (as they rely on
>>>> the old testsuite) and rather have them running on the
>>>> integration-arquillian testsuite. I know it sucks during development
>>>> that Jetty itself doesn't run in same JVM like the test (debugging
>>>> etc). But I hope we can have the solution, which will allow to
>>>> choose whether Jetty (or hopefully some other servers) are running
>>>> in same JVM or different JVM.
>>>>
>>> Unless arquillian can support running these adapter tests within an
>>> IDE, they need to stay.
>> Arquillian can support that. All we need is just different
>> implementations of arquillian DeployableContainer interface to
>> represent various containers. I've managed to have adapter tests
>> already running on embedded undertow and everything is running in
>> single JVM. Development and debugging is fine with that.
>>> In general, More often than not, I don't like the arquillian style
>>> we've enforced because it takes a lot longer to write tests, is often
>>> nearly impossible to do everything through a REST interface, and is
>>> often really hard to debug.  Maybe its just the types of things I'm
>>> testing.  Some of the tests should not have been ported to
>>> arquillian, or, at least shouldn't have been ported to using a REST
>>> interface for the entire test.
>> Yeah, having the test+server+adapter in single JVM just rocks during
>> development. Problem is, that it won't handle all type of errors. For
>> example if Jetty adapter ZIP distribution is broken (eg. some JAR file
>> is missing), the Jetty adapter tests, which uses embedded Jetty, won't
>> catch that. For those and similar reasons, also QA team needs to test
>> with "real" containers, not with the embedded ones.
>>
>> IMO ideal is, if we can have both possibilities. Test with embedded
>> container during development, but also has possibility to run the same
>> test against external container in different JVM. Arquillian testsuite
>> can support both approaches, but the old testsuite doesn't support the
>> external...
>>
>> I agree some tests shouldn't use REST endpoints. Like the model tests
>> from the old testsuite. But they still need to be able to run on
>> external environment though. I guess we can have something like the
>> "bridge" test, which will just deploy the model test to the REST
>> endpoint and you will be able to test things inside this REST endpoint
>> on server side? So you will be able to use all the stuff like model
>> classes etc.
>>
>> Marek
>>> Bill
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev