7:14 p.m.
We've picked Htmlunit as the default browser to run tests with due to it
being the fastest. Downside is that it simply doesn't work very well for
all tests, especially those heavy on the javascript side of things like
testing the JavaScript adapter and admin console.
Just saw that Chrome is actually bringing a headless option to Chrome 59
[1]. This is really nice as it allows headless testing with a real browser,
not just an emulated browser.
Ideally if this is fast we could use it as the default browser instead of
htmlunit. Obviously waiting until it's released on all platforms. If it's
not as fast as htmlunit then maybe there is a compromise.
The default browser would still be htmlunit. Then individual tests could be
marked (with an annotation on the class or on the WebDriver field). Those
marked would use Chrome in headless mode instead of htmlunit. Obviously
-Dbrowser would continue to override the browser in either case.
Thoughts? Anyone interested in giving the new headless Chrome option a spin
and evaluating how fast it is compared to htmlunit?
[1] https://developers.google.com/web/updates/2017/04/headless-chrome
4:53 a.m.
Just a note, the main reason why htmlUnit web driver is so faster than
other browsers is, that it is fully embedded in the same JVM like tests.
There is no additional overhead in the communication with other
processes. The only exception are the HTTP requests for the
communication with the tested Keycloak server and adapters.
On the other hand, the other web driver implementations, including
phantomJS, are using external process and there is remote selenium
server with which the webDriver needs to communicate. It means that all
the webDriver calls including the most simple (like driver.getTitle() )
need to send additional HTTP request to the remote selenium server under
the covers. All of this adds an additional overhead.
I suspect that headless chrome will also use remote selenium, hence I
don't expect that performance will be much better comparing to
phantomJS. But maybe I am wrong..
Marek
On 07/06/17 12:14, Stian Thorgersen wrote:
We've picked Htmlunit as the default browser to run tests with
due to it
being the fastest. Downside is that it simply doesn't work very well for
all tests, especially those heavy on the javascript side of things like
testing the JavaScript adapter and admin console.
Just saw that Chrome is actually bringing a headless option to Chrome 59
[1]. This is really nice as it allows headless testing with a real browser,
not just an emulated browser.
Ideally if this is fast we could use it as the default browser instead of
htmlunit. Obviously waiting until it's released on all platforms. If it's
not as fast as htmlunit then maybe there is a compromise.
The default browser would still be htmlunit. Then individual tests could be
marked (with an annotation on the class or on the WebDriver field). Those
marked would use Chrome in headless mode instead of htmlunit. Obviously
-Dbrowser would continue to override the browser in either case.
Thoughts? Anyone interested in giving the new headless Chrome option a spin
and evaluating how fast it is compared to htmlunit?
[1] https://developers.google.com/web/updates/2017/04/headless-chrome
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
3:01 p.m.
Is it always using HTTP? Surely it could be using some sort of sockets for
better perf. I think you're right though and we'll probably end up sticking
with htmlunit for most tests.
On 7 June 2017 at 21:53, Marek Posolda <mposolda(a)redhat.com> wrote:
Just a note, the main reason why htmlUnit web driver is so faster
than
other browsers is, that it is fully embedded in the same JVM like tests.
There is no additional overhead in the communication with other processes.
The only exception are the HTTP requests for the communication with the
tested Keycloak server and adapters.
On the other hand, the other web driver implementations, including
phantomJS, are using external process and there is remote selenium server
with which the webDriver needs to communicate. It means that all the
webDriver calls including the most simple (like driver.getTitle() ) need to
send additional HTTP request to the remote selenium server under the
covers. All of this adds an additional overhead.
I suspect that headless chrome will also use remote selenium, hence I
don't expect that performance will be much better comparing to phantomJS.
But maybe I am wrong..
Marek
On 07/06/17 12:14, Stian Thorgersen wrote:
> We've picked Htmlunit as the default browser to run tests with due to it
> being the fastest. Downside is that it simply doesn't work very well for
> all tests, especially those heavy on the javascript side of things like
> testing the JavaScript adapter and admin console.
>
> Just saw that Chrome is actually bringing a headless option to Chrome 59
> [1]. This is really nice as it allows headless testing with a real
> browser,
> not just an emulated browser.
>
> Ideally if this is fast we could use it as the default browser instead of
> htmlunit. Obviously waiting until it's released on all platforms. If it's
> not as fast as htmlunit then maybe there is a compromise.
>
> The default browser would still be htmlunit. Then individual tests could
> be
> marked (with an annotation on the class or on the WebDriver field). Those
> marked would use Chrome in headless mode instead of htmlunit. Obviously
> -Dbrowser would continue to override the browser in either case.
>
> Thoughts? Anyone interested in giving the new headless Chrome option a
> spin
> and evaluating how fast it is compared to htmlunit?
>
> [1] https://developers.google.com/web/updates/2017/04/headless-chrome
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
4:20 p.m.
I agree with Marek - I don't think it would do any good for us as the
default browser.
Imho the new headless mode doesn't make a much difference for us - the
Chrome would be working on the same principle as before (through
ChromeDriver).
Moreover, Chrome and ChromeDriver would need to be downloaded/installed
from external sources - yet, this could be perhaps solvable with Maven (at
least ChromeDriver as it's in the Maven repo).
On Wed, Jun 7, 2017 at 9:53 PM, Marek Posolda <mposolda(a)redhat.com> wrote:
Just a note, the main reason why htmlUnit web driver is so faster
than
other browsers is, that it is fully embedded in the same JVM like tests.
There is no additional overhead in the communication with other
processes. The only exception are the HTTP requests for the
communication with the tested Keycloak server and adapters.
On the other hand, the other web driver implementations, including
phantomJS, are using external process and there is remote selenium
server with which the webDriver needs to communicate. It means that all
the webDriver calls including the most simple (like driver.getTitle() )
need to send additional HTTP request to the remote selenium server under
the covers. All of this adds an additional overhead.
I suspect that headless chrome will also use remote selenium, hence I
don't expect that performance will be much better comparing to
phantomJS. But maybe I am wrong..
Marek
On 07/06/17 12:14, Stian Thorgersen wrote:
> We've picked Htmlunit as the default browser to run tests with due to it
> being the fastest. Downside is that it simply doesn't work very well for
> all tests, especially those heavy on the javascript side of things like
> testing the JavaScript adapter and admin console.
>
> Just saw that Chrome is actually bringing a headless option to Chrome 59
> [1]. This is really nice as it allows headless testing with a real
browser,
> not just an emulated browser.
>
> Ideally if this is fast we could use it as the default browser instead of
> htmlunit. Obviously waiting until it's released on all platforms. If it's
> not as fast as htmlunit then maybe there is a compromise.
>
> The default browser would still be htmlunit. Then individual tests could
be
> marked (with an annotation on the class or on the WebDriver field). Those
> marked would use Chrome in headless mode instead of htmlunit. Obviously
> -Dbrowser would continue to override the browser in either case.
>
> Thoughts? Anyone interested in giving the new headless Chrome option a
spin
> and evaluating how fast it is compared to htmlunit?
>
> [1] https://developers.google.com/web/updates/2017/04/headless-chrome
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
Václav Muzikář
Quality Engineer
Keycloak / Red Hat Single Sign-On
Red Hat Czech s.r.o.
5:27 p.m.
From what I see, there is no support for other protocol besides HTTP.
All the WebDriver implementations despite HtmlUnit extends from
RemoteWebDriver, which uses the CommandExecutor interface for
communicate with the server. In theory, the CommandExecutor
implementation is pluggable, but in reality, all the implementations
uses HTTP based command executor, which uses Apache HTTP Client under
the covers to communicate with the selenium server and they don't
provide straightforward way to use something different.
Not sure if there is some performance improvement in newest version? We
use the WebDriver version, which uses selenium 2.53.0. Now there is
already version 3 available.
Marek
On 08/06/17 09:20, Vaclav Muzikar wrote:
I agree with Marek - I don't think it would do any good for us as
the
default browser.
Imho the new headless mode doesn't make a much difference for us - the
Chrome would be working on the same principle as before (through
ChromeDriver).
Moreover, Chrome and ChromeDriver would need to be
downloaded/installed from external sources - yet, this could be
perhaps solvable with Maven (at least ChromeDriver as it's in the
Maven repo).
On Wed, Jun 7, 2017 at 9:53 PM, Marek Posolda <mposolda(a)redhat.com
<mailto:mposolda@redhat.com>> wrote:
Just a note, the main reason why htmlUnit web driver is so faster than
other browsers is, that it is fully embedded in the same JVM like
tests.
There is no additional overhead in the communication with other
processes. The only exception are the HTTP requests for the
communication with the tested Keycloak server and adapters.
On the other hand, the other web driver implementations, including
phantomJS, are using external process and there is remote selenium
server with which the webDriver needs to communicate. It means
that all
the webDriver calls including the most simple (like
driver.getTitle() )
need to send additional HTTP request to the remote selenium server
under
the covers. All of this adds an additional overhead.
I suspect that headless chrome will also use remote selenium, hence I
don't expect that performance will be much better comparing to
phantomJS. But maybe I am wrong..
Marek
On 07/06/17 12:14, Stian Thorgersen wrote:
> We've picked Htmlunit as the default browser to run tests with
due to it
> being the fastest. Downside is that it simply doesn't work very
well for
> all tests, especially those heavy on the javascript side of
things like
> testing the JavaScript adapter and admin console.
>
> Just saw that Chrome is actually bringing a headless option to
Chrome 59
> [1]. This is really nice as it allows headless testing with a
real browser,
> not just an emulated browser.
>
> Ideally if this is fast we could use it as the default browser
instead of
> htmlunit. Obviously waiting until it's released on all
platforms. If it's
> not as fast as htmlunit then maybe there is a compromise.
>
> The default browser would still be htmlunit. Then individual
tests could be
> marked (with an annotation on the class or on the WebDriver
field). Those
> marked would use Chrome in headless mode instead of htmlunit.
Obviously
> -Dbrowser would continue to override the browser in either case.
>
> Thoughts? Anyone interested in giving the new headless Chrome
option a spin
> and evaluating how fast it is compared to htmlunit?
>
> [1]
https://developers.google.com/web/updates/2017/04/headless-chrome
<https://developers.google.com/web/updates/2017/04/headless-chrome>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
<https://lists.jboss.org/mailman/listinfo/keycloak-dev>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
<https://lists.jboss.org/mailman/listinfo/keycloak-dev>
--
Václav Muzikář
Quality Engineer
Keycloak / Red Hat Single Sign-On
Red Hat Czech s.r.o.
9:58 p.m.
Ok, so default browser will 99.999% remain htmlunit. That means we do need
an option to specify two browsers. First browser defaults to htmlunit,
second browser needs to default to something that works well with JS
testing. The second browser could default to Chrome headless, but would be
possible to disable and in that case those particular tests wouldn't run. I
just don't think it's going to be feasible to get all tests to run on
htmlunit. Feel free to correct me if I'm wrong on the latter part and it is
indeed possible to test everything with htmlunit.
On 8 June 2017 at 10:27, Marek Posolda <mposolda(a)redhat.com> wrote:
From what I see, there is no support for other protocol besides HTTP.
All
the WebDriver implementations despite HtmlUnit extends from
RemoteWebDriver, which uses the CommandExecutor interface for communicate
with the server. In theory, the CommandExecutor implementation is
pluggable, but in reality, all the implementations uses HTTP based command
executor, which uses Apache HTTP Client under the covers to communicate
with the selenium server and they don't provide straightforward way to use
something different.
Not sure if there is some performance improvement in newest version? We
use the WebDriver version, which uses selenium 2.53.0. Now there is already
version 3 available.
Marek
On 08/06/17 09:20, Vaclav Muzikar wrote:
I agree with Marek - I don't think it would do any good for us as the
default browser.
Imho the new headless mode doesn't make a much difference for us - the
Chrome would be working on the same principle as before (through
ChromeDriver).
Moreover, Chrome and ChromeDriver would need to be downloaded/installed
from external sources - yet, this could be perhaps solvable with Maven (at
least ChromeDriver as it's in the Maven repo).
On Wed, Jun 7, 2017 at 9:53 PM, Marek Posolda <mposolda(a)redhat.com> wrote:
> Just a note, the main reason why htmlUnit web driver is so faster than
> other browsers is, that it is fully embedded in the same JVM like tests.
> There is no additional overhead in the communication with other
> processes. The only exception are the HTTP requests for the
> communication with the tested Keycloak server and adapters.
>
> On the other hand, the other web driver implementations, including
> phantomJS, are using external process and there is remote selenium
> server with which the webDriver needs to communicate. It means that all
> the webDriver calls including the most simple (like driver.getTitle() )
> need to send additional HTTP request to the remote selenium server under
> the covers. All of this adds an additional overhead.
>
> I suspect that headless chrome will also use remote selenium, hence I
> don't expect that performance will be much better comparing to
> phantomJS. But maybe I am wrong..
>
> Marek
>
> On 07/06/17 12:14, Stian Thorgersen wrote:
> > We've picked Htmlunit as the default browser to run tests with due to it
> > being the fastest. Downside is that it simply doesn't work very well for
> > all tests, especially those heavy on the javascript side of things like
> > testing the JavaScript adapter and admin console.
> >
> > Just saw that Chrome is actually bringing a headless option to Chrome 59
> > [1]. This is really nice as it allows headless testing with a real
> browser,
> > not just an emulated browser.
> >
> > Ideally if this is fast we could use it as the default browser instead
> of
> > htmlunit. Obviously waiting until it's released on all platforms. If
> it's
> > not as fast as htmlunit then maybe there is a compromise.
> >
> > The default browser would still be htmlunit. Then individual tests
> could be
> > marked (with an annotation on the class or on the WebDriver field).
> Those
> > marked would use Chrome in headless mode instead of htmlunit. Obviously
> > -Dbrowser would continue to override the browser in either case.
> >
> > Thoughts? Anyone interested in giving the new headless Chrome option a
> spin
> > and evaluating how fast it is compared to htmlunit?
> >
> > [1] https://developers.google.com/web/updates/2017/04/headless-chrome
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
--
Václav Muzikář
Quality Engineer
Keycloak / Red Hat Single Sign-On
Red Hat Czech s.r.o.
7:08 p.m.
Have you guys looked at Drone lately? There is a bunch of features you
might like, such as an automatic download of drivers [1] and defining
target browser through a simple property. As for headless chrome, we are
almost there [2]
HTH
Bartosz
[1] http://arquillian.org/arquillian-extension-drone/#_automatic_download
[2] https://github.com/arquillian/arquillian-extension-drone/pull/92
On Thu, Jun 8, 2017 at 2:58 PM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
Ok, so default browser will 99.999% remain htmlunit. That means we do
need
an option to specify two browsers. First browser defaults to htmlunit,
second browser needs to default to something that works well with JS
testing. The second browser could default to Chrome headless, but would be
possible to disable and in that case those particular tests wouldn't run. I
just don't think it's going to be feasible to get all tests to run on
htmlunit. Feel free to correct me if I'm wrong on the latter part and it is
indeed possible to test everything with htmlunit.
On 8 June 2017 at 10:27, Marek Posolda <mposolda(a)redhat.com> wrote:
> From what I see, there is no support for other protocol besides HTTP. All
> the WebDriver implementations despite HtmlUnit extends from
> RemoteWebDriver, which uses the CommandExecutor interface for communicate
> with the server. In theory, the CommandExecutor implementation is
> pluggable, but in reality, all the implementations uses HTTP based
command
> executor, which uses Apache HTTP Client under the covers to communicate
> with the selenium server and they don't provide straightforward way to
use
> something different.
>
> Not sure if there is some performance improvement in newest version? We
> use the WebDriver version, which uses selenium 2.53.0. Now there is
already
> version 3 available.
>
> Marek
>
>
> On 08/06/17 09:20, Vaclav Muzikar wrote:
>
> I agree with Marek - I don't think it would do any good for us as the
> default browser.
> Imho the new headless mode doesn't make a much difference for us - the
> Chrome would be working on the same principle as before (through
> ChromeDriver).
> Moreover, Chrome and ChromeDriver would need to be downloaded/installed
> from external sources - yet, this could be perhaps solvable with Maven
(at
> least ChromeDriver as it's in the Maven repo).
>
> On Wed, Jun 7, 2017 at 9:53 PM, Marek Posolda <mposolda(a)redhat.com>
wrote:
>
>> Just a note, the main reason why htmlUnit web driver is so faster than
>> other browsers is, that it is fully embedded in the same JVM like tests.
>> There is no additional overhead in the communication with other
>> processes. The only exception are the HTTP requests for the
>> communication with the tested Keycloak server and adapters.
>>
>> On the other hand, the other web driver implementations, including
>> phantomJS, are using external process and there is remote selenium
>> server with which the webDriver needs to communicate. It means that all
>> the webDriver calls including the most simple (like driver.getTitle() )
>> need to send additional HTTP request to the remote selenium server under
>> the covers. All of this adds an additional overhead.
>>
>> I suspect that headless chrome will also use remote selenium, hence I
>> don't expect that performance will be much better comparing to
>> phantomJS. But maybe I am wrong..
>>
>> Marek
>>
>> On 07/06/17 12:14, Stian Thorgersen wrote:
>> > We've picked Htmlunit as the default browser to run tests with due to
it
>> > being the fastest. Downside is that it simply doesn't work very well
for
>> > all tests, especially those heavy on the javascript side of things
like
>> > testing the JavaScript adapter and admin console.
>> >
>> > Just saw that Chrome is actually bringing a headless option to Chrome
59
>> > [1]. This is really nice as it allows headless testing with a real
>> browser,
>> > not just an emulated browser.
>> >
>> > Ideally if this is fast we could use it as the default browser instead
>> of
>> > htmlunit. Obviously waiting until it's released on all platforms. If
>> it's
>> > not as fast as htmlunit then maybe there is a compromise.
>> >
>> > The default browser would still be htmlunit. Then individual tests
>> could be
>> > marked (with an annotation on the class or on the WebDriver field).
>> Those
>> > marked would use Chrome in headless mode instead of htmlunit.
Obviously
>> > -Dbrowser would continue to override the browser in either case.
>> >
>> > Thoughts? Anyone interested in giving the new headless Chrome option a
>> spin
>> > and evaluating how fast it is compared to htmlunit?
>> >
>> > [1] https://developers.google.com/web/updates/2017/04/headless-chrome
>> > _______________________________________________
>> > keycloak-dev mailing list
>> > keycloak-dev(a)lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>
>
>
> --
> Václav Muzikář
> Quality Engineer
> Keycloak / Red Hat Single Sign-On
> Red Hat Czech s.r.o.
>
>
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
2526
days inactive
2528
days old
6 comments
4 participants
participants (4)
-
Bartosz Majsak -
Marek Posolda -
Stian Thorgersen -
Vaclav Muzikar