[keycloak-dev] Javascript adapater tests

Thu Jan 17 05:01:05 EST 2019

Unit tests can be helpful during development and especially refactoring.
However, they do not prove that it is working until you have integration
tests. Unit tests can be added for sure, but we can't accept a larger
rewrite of the adapter until there is proper integration testing in place.
There is some coverage today, but it is not good enough to do a big rewrite.

The code itself has grown to a point where it is becoming hard to manage.
However, you are over stressing this point. We have a large number of happy
users of the adapters as well as many contributors that have had no issue
in contributing to the code. All in all it is not that complicated and
there are tradeoffs with bringing in Typescript, modules, etc. that may
just not be worth the effort for what is in essence a fairly simple library.

On Thu, 17 Jan 2019 at 10:41, Guillaume Vincent <gvincent at redhat.com> wrote:

> Hello,
>
> > With regards to testing I see no value in unit tests for the JavaScript
> adapter.
>
> Without unit and functional tests, JavaScript developers can't refactor
> the internal structure of the code.
> Testing is about feedback, and the feedback loop must be short.
> Integration tests take a long time to write, maintain and run.
> Integration tests are needed, but also and above all, we need automated
> and fast javascript tests.
>
> Once again, I do not think we have the same vision of what the Javascript
> adapter should be.
> I think we have a fundamental problem.
>
> When I see the size and quality of the code, the number of bugs in all the
> wrappers for each framework, the time it takes to fix simple bugs, the
> complicated use of the library.
> This make me sad, and I loose a lot of time using this library.
>
> > I am open to suggestions on how to write integration tests with
> JavaScript frameworks
>
> I let you do it, I think you are going in the wrong direction. Tell me
> when you have changed your mind.
>
> Cheers
>
>
> On Thu, Jan 17, 2019 at 10:22 AM Stian Thorgersen <sthorger at redhat.com>
> wrote:
>
>> With regards to converting to TypeScript I agree that is the way forward.
>>
>> With regards to testing I see no value in unit tests for the JavaScript
>> adapter. The adapter itself provides very little logic and it's mostly
>> about invoking the Keycloak server. As a rule Keycloak requires
>> functional/integration tests, while unit tests are optional.
>>
>> I am open to suggestions on how to write integration tests with
>> JavaScript frameworks instead of Arquillian. Couple things to point out. It
>> has to be integration tests using a real Keycloak server and it has to
>> support browser testing using Sellenium.
>>
>>
>> On Wed, 16 Jan 2019 at 20:36, Bruno Oliveira <bruno at abstractj.org> wrote:
>>
>>> That's great Guillaume, and I agree with Stan about TypeScript. One
>>> thing to keep in mind is the fact that most of our tests today are
>>> integration tests.
>>>
>>> Stan knows more than me about this. But talking about what was done
>>> for the Node.js adapter[1], what we did was to write an app[2] which
>>> was able to "exercise" some parts of the code.
>>>
>>> Maybe we can do something similar for the JavaScript adapter? If this
>>> is something which you would like to take a look, I'd suggest to
>>> create an enhancement into Jira. In this way people will be aware that
>>> you're on it.
>>>
>>> [1] - https://github.com/keycloak/keycloak-nodejs-connect
>>> [2] -
>>> https://github.com/keycloak/keycloak-nodejs-connect/tree/master/test/fixtures
>>>
>>> On Wed, Jan 16, 2019 at 4:16 PM Guillaume Vincent <gvincent at redhat.com>
>>> wrote:
>>> >
>>> > > Coverage is what matters.
>>> >
>>> > I like to tests features and add regression tests for each bugs.
>>> > Coverage is not a good metric, you can call your function with no
>>> > assertion, you will have 100% code coverage.
>>> > But I think we are agree that every function in the adapter (init(),
>>> > login() refreshToken(), etc) should be tested.
>>> >
>>> > > should be written in TypeScript.
>>> >
>>> > Totally agree, as I said I created the experience in a couple of hours.
>>> > But yes JS adapter should be written in typescript
>>> >
>>> > I see 2 strategies to implement tests and upgrade the JS adapter:
>>> >
>>> >  1/ write a new lib in Typescript near the old one, add some tests for
>>> > every function and test new and old implementation together with
>>> automated
>>> > tests.
>>> >  2/ split actual code (each method in a file for example) set up a
>>> blunder
>>> > to build the js adapter and then add unit and functional tests one by
>>> one
>>> > for every function. Then migrate code to Typescript
>>> >
>>> > I can try a POC on the first one if you want
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Jan 16, 2019 at 5:43 PM Stan Silvert <ssilvert at redhat.com>
>>> wrote:
>>> >
>>> > > On 1/16/2019 9:53 AM, Guillaume Vincent wrote:
>>> > > > Hello Keycloak dev list,
>>> > > >
>>> > > > in a previous post I raised the problem that the JavaScript
>>> adapter did
>>> > > not
>>> > > > have JavaScript tests.
>>> > > >
>>> > > > In a couple of hours I created a simple example for Keycloak with
>>> unit
>>> > > and
>>> > > > functional tests https://github.com/guillaumevincent/keycloak-lite
>>> > > >
>>> > > > You can see tests in this file
>>> > > >
>>> https://github.com/guillaumevincent/keycloak-lite/blob/master/test.js
>>> > > >
>>> > > > I also created a blog post on IMO How to test JavaScript code:
>>> > > > https://guillaumevincent.com/2019/01/15/test-in-javascript.html
>>> > > >
>>> > > > Maybe we can open the discussion on how keycloak.js should be
>>> tested.
>>> > > > Without any fast and automated tests, in JavaScript, the refactor
>>> of the
>>> > > > keycloak adapter will not be easy at all.
>>> > > >
>>> > > > wdyt?
>>> > > Several thoughts:
>>> > >
>>> > >   * Basically, I agree.  It makes sense to test javascript with
>>> > >     javascript.  I like where you are going with this.
>>> > >   * An important point in any discussion of testing is that the only
>>> > >     useful test is a test that uncovers a bug.  We never write tests
>>> > >     just to say we have lots of tests.  Coverage is what matters.
>>> I'm
>>> > >     not criticizing your blog.  It's just something I like to keep
>>> in mind.
>>> > >   * You mention TypeScript in your blog, but test.js appears to be
>>> > >     written in plain javascript.  IMO, any javascript we write (with
>>> the
>>> > >     possible exception of keycloak.js) should be written in
>>> TypeScript.
>>> > >     Both internally and externally, developers are moving more and
>>> more
>>> > >     to TypeScript.  Also, the Java developers on our team will be
>>> much
>>> > >     more comfortable and confident with a strongly typed language
>>> that
>>> > >     works well with an IDE.
>>> > >   * We need to know a little more about the current test coverage of
>>> the
>>> > >     javascript adapter.  Much of it is tested through indirect means.
>>> > >   * We need to understand how javascript tests will integrate into
>>> our
>>> > >     builds.
>>> > >   * We need to standardize on a javascript test package.  I don't
>>> want
>>> > >     the adapter to be tested with one library while the new account
>>> > >     management console is tested with another.
>>> > >
>>> > >
>>> > >
>>> > > _______________________________________________
>>> > > keycloak-dev mailing list
>>> > > keycloak-dev at lists.jboss.org
>>> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> >
>>> >
>>> >
>>> > --
>>> > Guillaume Vincent
>>> > Senior Software Engineer
>>> > _______________________________________________
>>> > keycloak-dev mailing list
>>> > keycloak-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>>
>>>
>>> --
>>> - abstractj
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>
>
> --
> Guillaume Vincent
> Senior Software Engineer
>