Hi Thomas,

That's great news, thanks for sharing. We've tried to execute these tests a while back, but there was issues with them at the time. Our plan is to revisit this in the next few months and to resolve issues where we're not following the spec.

On 25 May 2016 at 00:03, Thomas Darimont <thomas.darimont@googlemail.com> wrote:
Hello list,

sorry for the longer email...

I just noticed that Keycloak is currently not listed as a 
certified OpenID Connect implementation under:
http://openid.net/certification/

As it turns out one can run the tests oneself by creating a test profile as described here:
http://openid.net/certification/testing/

The OpenID Connect test can be configured here: 
https://op.certification.openid.net:60000/

I just gave the test a spin by running a Keycloak Application instance 
(Version 1.9.1.Final - as I had that running) embedded in a Spring Boot App 
on Cloud Foundry which I exposed to the op.certification.openid.net test server.
... it works and was a quick way to get Keycloak exposed to the test - and yes I know 
this is of course not a prod environment ;-)

The results looked not bad.
Note that you need to execute each step manually by clicking on it... 

First run got me 23 green (+2 manually verified) out of 41 tests overall,
rest was 9 yellow and 6 red.

You can find a screenshot of the overall test results here:

I think those tests are a great way to close gaps between specification and implementation 
and help to make Keycloak more compatible.

I also have the logs with the detailed request / response pairs with failed tests and 
explanations.
Please ping me if you want to have those for investigation  (~600 kb text).

Some of the tests like ("Scope requesting all claims [Basic, Implicit, Hybrid] (OP-scope-All)") 
were yellow because the some claim information was missing in the user info like:
['nickname', 'profile', 'picture', 'website', 'gender', 'birthdate', 'zoneinfo', 'locale', 'updated_at', 'phone_number', 'phone_number_verified'].

The red tests like "IDToken has kid [Basic, Implicit, Hybrid] (OP-IDToken-kid)" mostly failed due to 
missing values in the response e.g. 
"[verify-signed-idtoken-has-kid]
status: ERROR
description: Verifies that the header of a signed IDToken includes a kid claim.
info: Signed ID Token has no kid: header={u'alg': u'RS256'}"

If you want to try it out yourself here are the settings I used for the 
OpenID Connect Test Application:

--------------------

Provider configuration:
"Does the OP have a .well-known/openid-configuration endpoint?"
yes

"What is the issuer path for this configuration information?"
https://tdlabs-keycloak-test2.cfapps.io/realms/test

"Do the provider support dynamic client registration?"
no (I know keycloak supports that but I couldn't get that working)

"Redirect uris"
https://op.certification.openid.net:60629/authz_cb

"Client id" 
openid-cert

"Client secret"
4692ca28-daad-4d76-aa82-0991e518d931

Required info
"Which subject type do you want to use by default?"
public 

"Which response type should be used by default?"
code

"Select supported features"
JWT signed with algorithm other than "none"
Encrypted JWT

Test specific request parameters:

"Login hint"
tom@example.com
"UI locales"
en de
"Claims locales"
en de
"Acr values"
2 1

"Webfinger url"
https://example.com/tom

"Webfinger email"
tom@example.com
E.g. bob@example.com

For testing purposes I created a new realm "test" with an additional 
client "openid-cert" with "confidential" access type and 
the valid redirect url provided by the op.certification.openid.net test server.

I also created a user "tester" for the login tests.

Cheers,
Thomas

_______________________________________________
keycloak-dev mailing list
keycloak-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev