[keycloak-dev] OpenID Connect Certification Tests

[Thomas Darimont]

Hello list,

sorry for the longer email...

I just noticed that Keycloak is currently not listed as a
certified OpenID Connect implementation under:
http://openid.net/certification/

As it turns out one can run the tests oneself by creating a test profile as
described here:
http://openid.net/certification/testing/

The OpenID Connect test can be configured here:
https://op.certification.openid.net:60000/

I just gave the test a spin by running a Keycloak Application instance
(Version 1.9.1.Final - as I had that running) embedded in a Spring Boot App
on Cloud Foundry which I exposed to the op.certification.openid.net test
server.
... it works and was a quick way to get Keycloak exposed to the test - and
yes I know
this is of course not a prod environment ;-)

The results looked not bad.
Note that you need to execute each step manually by clicking on it...

First run got me 23 green (+2 manually verified) out of 41 tests overall,
rest was 9 yellow and 6 red.

You can find a screenshot of the overall test results here:
http://s33.postimg.org/h6zawnbbz/screencapture_op_certification_openid_net_60628.png

I think those tests are a great way to close gaps between specification and
implementation
and help to make Keycloak more compatible.

I also have the logs with the detailed request / response pairs with failed
tests and
explanations.
Please ping me if you want to have those for investigation  (~600 kb text).

Some of the tests like ("Scope requesting all claims [Basic, Implicit,
Hybrid] (OP-scope-All)")
were yellow because the some claim information was missing in the user info
like:
['nickname', 'profile', 'picture', 'website', 'gender', 'birthdate',
'zoneinfo', 'locale', 'updated_at', 'phone_number',
'phone_number_verified'].

The red tests like "IDToken has kid [Basic, Implicit, Hybrid]
(OP-IDToken-kid)" mostly failed due to
missing values in the response e.g.
"[verify-signed-idtoken-has-kid]
status: ERROR
description: Verifies that the header of a signed IDToken includes a kid
claim.
info: Signed ID Token has no kid: header={u'alg': u'RS256'}"

If you want to try it out yourself here are the settings I used for the
OpenID Connect Test Application:

--------------------

Provider configuration:
"Does the OP have a .well-known/openid-configuration endpoint?"
yes

"What is the issuer path for this configuration information?"
https://tdlabs-keycloak-test2.cfapps.io/realms/test

"Do the provider support dynamic client registration?"
no (I know keycloak supports that but I couldn't get that working)

"Redirect uris"
https://op.certification.openid.net:60629/authz_cb

"Client id"
openid-cert

"Client secret"
4692ca28-daad-4d76-aa82-0991e518d931

Required info
"Which subject type do you want to use by default?"
public

"Which response type should be used by default?"
code

"Select supported features"
JWT signed with algorithm other than "none"
Encrypted JWT

Test specific request parameters:

"Login hint"
tom at example.com
"UI locales"
en de
"Claims locales"
en de
"Acr values"
2 1

"Webfinger url"
https://example.com/tom

"Webfinger email"
tom at example.com
E.g. bob at example.com

For testing purposes I created a new realm "test" with an additional
client "openid-cert" with "confidential" access type and
the valid redirect url provided by the op.certification.openid.net test
server.

I also created a user "tester" for the login tests.

Cheers,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160525/5ccfaa7f/attachment.html