The UI tests are not supposed to work with "artificial" browsers - only the real-life ones are supported. UI tests are high-demanding on JS interpreter (PhantomJS nor HtmlUnit can satisfy such demands) and moreover UI needs to be anyway tested with all supported browsers (due to some differences between them). So it wouldn't make much sense to run them with e.g. HtmlUnit. Please see our HOW-TO-RUN [1] where are the instructions on some special tests (like UI or adapters). However, good point that we should add some check for supported browsers before running the tests. I'll look into it. [1] https://github.com/keycloak/keycloak/blob/master/ testsuite/integration-arquillian/HOW-TO-RUN.md#ui-tests V. On Wed, May 31, 2017 at 8:41 AM, Pavel Drozd <pdrozd(a)redhat.com&gt; wrote: > Dne 31.5.2017 v 08:31 Stian Thorgersen napsal(a): > > Ideally the console tests should work with HtmlUnit, but failing that > they certainly need to work with PhantomJS. > > Pavel - any chance we can get the console tests working with HtmlUnit? If > not I guess we need to set the default for the console tests to PhantomJS. > > > We were focusing to run the tests mainly with real browsers. Vasek did > you try to run the UI tests with HtmlUnit? > > > On 30 May 2017 at 17:37, Alex Szczuczko <aszczucz(a)redhat.com&gt; wrote: > >> Hi, >> >> I just lost a couple days to a really simple mistake when running the UI >> tests. I didn't set -Dbrowser=firefox, so the headless WebDriver was used, >> which produces a lot of impossible errors that I couldn't figure out at >> all. To save the next person the same agony, I think there are two possible >> solutions: >> >> 1. Define a default browser=firefox property in >> testsuite/integration-arquillian/tests/other/console/pom.xml >> >> 2. Use a plugin to fail the build if the user has not defined the >> browser property on the command line. >> >> Note that you need to enable a profile to run these tests >> (-Pconsole-ui-tests), so this doesn't impact the project's normal mvn clean >> install. >> >> Opinions on either of these options? >> >> Thanks, >> Alex >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > -- Václav Muzikář Quality Engineer Keycloak / Red Hat Single Sign-On Red Hat Czech s.r.o.

My 2 cents - Travis can run tests with Chrome and Firefox [1] so there might be a way for automating these. +1 for checking whether the browser is supported or not before running the UI tests. --Hynek [1] https://docs.travis-ci.com/user/gui-and-headless-browsers/ On Thu, Jun 1, 2017 at 10:05 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > As long as the tests require a real browser we can't run them as part of a > default build and I also doubt we can run them on Travis. That's a problem > IMO. > > On 31 May 2017 at 18:43, Vaclav Muzikar <vmuzikar(a)redhat.com&gt; wrote: > >> The UI tests are not supposed to work with "artificial" browsers - only >> the real-life ones are supported. >> UI tests are high-demanding on JS interpreter (PhantomJS nor HtmlUnit can >> satisfy such demands) and moreover UI needs to be anyway tested with all >> supported browsers (due to some differences between them). So it wouldn't >> make much sense to run them with e.g. HtmlUnit. >> Please see our HOW-TO-RUN [1] where are the instructions on some special >> tests (like UI or adapters). >> >> However, good point that we should add some check for supported browsers >> before running the tests. I'll look into it. >> >> [1] https://github.com/keycloak/keycloak/blob/master/ >> testsuite/integration-arquillian/HOW-TO-RUN.md#ui-tests >> >> V. >> >> On Wed, May 31, 2017 at 8:41 AM, Pavel Drozd <pdrozd(a)redhat.com&gt; wrote: >> >>> Dne 31.5.2017 v 08:31 Stian Thorgersen napsal(a): >>> >>> Ideally the console tests should work with HtmlUnit, but failing that >>> they certainly need to work with PhantomJS. >>> >>> Pavel - any chance we can get the console tests working with HtmlUnit? If >>> not I guess we need to set the default for the console tests to PhantomJS. >>> >>> >>> We were focusing to run the tests mainly with real browsers. Vasek did >>> you try to run the UI tests with HtmlUnit? >>> >>> >>> On 30 May 2017 at 17:37, Alex Szczuczko <aszczucz(a)redhat.com&gt; wrote: >>> >>>> Hi, >>>> >>>> I just lost a couple days to a really simple mistake when running the UI >>>> tests. I didn't set -Dbrowser=firefox, so the headless WebDriver was used, >>>> which produces a lot of impossible errors that I couldn't figure out at >>>> all. To save the next person the same agony, I think there are two possible >>>> solutions: >>>> >>>> 1. Define a default browser=firefox property in >>>> testsuite/integration-arquillian/tests/other/console/pom.xml >>>> >>>> 2. Use a plugin to fail the build if the user has not defined the >>>> browser property on the command line. >>>> >>>> Note that you need to enable a profile to run these tests >>>> (-Pconsole-ui-tests), so this doesn't impact the project's normal mvn clean >>>> install. >>>> >>>> Opinions on either of these options? >>>> >>>> Thanks, >>>> Alex >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >>> >> >> >> -- >> Václav Muzikář >> Quality Engineer >> Keycloak / Red Hat Single Sign-On >> Red Hat Czech s.r.o. >> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -- --Hynek