[keycloak-dev] Performance concerns and improvements for large numbers of clients [KEYCLOAK-8275]

[Stian Thorgersen]

On Wed, 18 Sep 2019, 18:11 Cristian Schuszter, <cristian.schuszter at cern.ch>
wrote:

> Hello everyone,
>
> My team and I have been revisiting some performance issues, namely the
> ones related to: https://issues.jboss.org/browse/KEYCLOAK-8275 .
>
> >From the comments, I understand that the plan is to re-implement the data
> access part of Keycloak, which is why the issues have been closed or put
> back into triage. We are planning to roll out a production Keycloak
> instance in the coming months / weeks and we've been doing some statistics
> on the number of clients that we would need to support: somewhere in the
> region of 15 000.
>
> Firstly, I'd like to know what the status on the new / improved data
> access is, as I couldn't find any information on your Jira. Could you give
> us an estimate on when it would be released?
>
> Secondly, assuming that it will take more time until such a high number of
> clients will be supported fully, I have a few simple improvements or
> changes in mind, and we'd gladly contribute with a PR. They are as follows:
>
> 1. New query parameters for the /auth/admin/realms/:realmId/clients
> endpoint:
>   - limit (int): restrict the number of results when pulling all clients,
> useful for the admin UI as there's no point in pulling 200 pages of stuff,
> nobody's going to click the arrows to search manually
>

It should support pagination, not a limit. See other endpoints as it should
be consistent to what we have elsewhere.

  - search (bool): if set to true, searches with a case-insensitive "LIKE
> %clientId%" query, as currently the search box on the admin UI works only
> in the case of an exact match (not particularly useful).
>

Search at the moment is client-side (should be non-exact match I believe),
but yes to support pagination on server side the search has to be server
side and should be a fuzzy search like user searches.


>   The search flag combined with the limited results will allow admins to
> search through clients without the need to pull all the data from the
> server, causing a timeout. Any queries performed like before work the same,
> listing all clients.
>
> 2. Removal of "RedirectUtils.getValidRedirectUris()" from the class
> LogoutEndpoint (possibly turned on or off via a config flag??). With 15000
> clients it took around 5 minutes for the logout endpoint to return
> successfully. Without the redirect URI validation, it finished in
> milliseconds. Since any redirect URI of any client of the realm can be
> used, I don't personally see much use from using it at all. ( see
> https://issues.jboss.org/browse/KEYCLOAK-8284 )
>

This one is a bit trickier. Ideally it should follow the behaviour from
oidc session management spec. Clients should have
separate post_logout_redirect_uris registered. post_logout_redirect_uri
should only redirect if id_token_hint is included so we can identify the
client and Uri is valid. Otherwise we should just show a page with a link
requiring user to manually click the link.

However, that would be breaking behaviour for existing clients/user so we
would need some backwards compatibility mode. Could be done with an option
on the realm.

This one is probably worth having a separate thread on though.


> 3. Minor UI tweak: When pressing enter or clicking on search in the admin
> client list, a "Searching..." prompt appears under the seearch box.
>

Not sure what you mean. It did before or it does the way you've changed it
to?


  !! Talking about UI, I saw a strange thing in the Angular code, it seems
> that ALL the clients are pulled from the server when accessing the client
> details. I added a limit to that as well as they didn't seem to be used. I
> don't think the client list should be pulled by anything, but maybe
> somebody knows details here.
>

I suspect it may have something to do with scope as it shows a list of
clients, which also reminds me that roles have support for composite roles
which also lists all clients to pick client roles.


>
> The PR is submitted at https://github.com/keycloak/keycloak/pull/6320,
> any feedback would be appreciated!
>
> Best regards,
> Cristian Schuszter
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>