[keycloak-dev] WebAuthn testing

[Yoshikazu Nojima]

Hello Marek,

I'm Noji, main author of WebAuthn4J library.

It is webauthn library's responsibility to properly interpret the
WebAuthn request and validate it in the way defined in the
specification.
WebAuthn4J will do its best.
In order to ensure conformance with the specifications, we check
WebAuthn4J pass the FIDO Alliance conformance test tool.

## (a) Chrome Testing API / (b) WebDriver Authenticator Extension

I suggests WebAuthn4J library caller like KeyCloak to put a weight
caller/WebAuthn4J integration tests or E2E tests.
>From the perspective, using the Chrome Testing API is a good option,
and if circumstance allows, WebDriver Authenticator Extension would be
great as it can test client side javascript part.

## (c) FIDO Alliance conformance tool

FIDO Alliance conformance tool is an official test suite and has good
test cases, but it is Electron based desktop tool.
It cannot be fit in CI environment. I execute the tool manually in my laptop.

## (d) how WebAuthn4J library do

WebAuthn4J has unit tests per classes, which is executed in the CI.
FIDO Alliance conformance tool is also executed, but the execution is
not automated.
In order to increase the test coverage in CI, I'm developing
authenticator emulator (webauthn4j-test) in Java, and WebAuthn4J use
it for integration test.
ex.
https://github.com/webauthn4j/webauthn4j/blob/master/webauthn4j-core/src/test/java/integration/scenario/FIDOU2FAuthenticatorRegistrationValidationTest.java#L82-L95

authenticator emulator (webauthn4j-test) is still in development and
focusing on internal use for now, but it might be good option when it
finished.

Regards, Noji

2019年8月20日(火) 22:10 Marek Posolda <mposolda at redhat.com>:
>
> We have community PR for adding initial webauthn support [1]. The PR is
> great work for the start IMO (anyone welcome to review PR :)
>
> But among other things, it touches one of the challenges for webauthn
> support, which is automated testing. So far, the PR introduces testing
> webauthn with the Chrome WebDriver and the Chrome testing API [2] . The
> Chrome Testing API allows chrome browser to "simulate" the webauthn
> authenticator without having the real webauthn authenticator device
> available.
>
> IMO it is fine to use the Chrome Testing API for the parts, which are
> more focus on the Keycloak integration itself rather than on the details
> of WebAuthn configurations. From one point of view, it may be sufficient
> to test just with Chrome Testing API as Keycloak is just WebAuthn
> Relying party and it doesn't need to care which underlying WebAuthn
> authenticator was used - that's more the responsibilty of the platform
> and web browser, which sends HTTP requests to Keycloak. However I afraid
> that there may be some cases where this is not sufficient and we may
> need some more proper interoperability testing. Especially for testing
> the details of webauthn specification. For example if Keycloak sends the
> WebAuthn registration request with "userVerification: required", we may
> need to ensure that authentication will fail if userVerification flag is
> not provided by the authenticator etc.
>
> So what are possible options?
>
> (a) Use just testing with chrome testing API and rely on the underlying
> webauthn library (webauthn4j) to hide the webauthn specification
> interoperability testing. This may be the easy option, but not sure
> whether sufficient?
>
> (b) Look at WebDriver Authenticator Extension and check if/when it is
> possible to use it. Details here [2] (also see the open PR linked from
> there). I am not sure when this will be available for us to be able to
> consume it, so not sure if this is an option...
>
> (c) Look at FIDO Alliance certification conformance [3] and investigate
> if it provides some automated tests, which can be somehow
> used/forked/duplicated to our testsuite
>
> (d) Check how webauthn4j library (and eventually some other libraries)
> are doing automated tests and inspire from them if possible.
>
> (e) Test manually with some defined set of webauthn authenticators on
> some defined set of platforms/browsers. Then investigate which HTTP
> requests were sent to Keycloak from the browser for various
> configuration options and use those manual tests to create some
> "client", which will just use Apache HTTP Client under the covers to
> directly send HTTP requests to Keycloak. In other words, just simulate
> the browser+OS+Authenticator. This is not ideal, however possibly useful
> as a fallback option?
>
> (f) any better option?
>
> [1] https://github.com/keycloak/keycloak/pull/6248/files
>
> [2]
> https://chromium.googlesource.com/chromium/src/+/a82bdb76536b5fce185734225128ca3ccb5a2e8c
>
> [3] https://github.com/w3c/webauthn/issues/1236
>
> [4]
> https://fidoalliance.org/certification/functional-certification/conformance/
>
> Marek
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev