[keycloak-dev] large number of realms causing slow api calls

[Gideon Caranzo]

Thank Stian for your reply. API calls have improved after using a different
composite role (with few realm roles).

Aside from API calls, I also observed slow startup time (about 20 mins). I
found the following calls during startup is taking more time.

*First*, the check for new installation using
applianceBootstrap.isNewInstall() at
KeycloakApplication.migrateAndBootstrap() is causing all realms to be
queried.

    public boolean isNewInstall() {
        if (session.realms().getRealms().size() > 0) {
            return false;
        } else {
            return true;
        }
    }

A count query will make this faster. So the condition can be something
like: if (session.realms().getRealmCount() > 0)

*Second*, call to UserStorageSyncManager.bootstrapPeriodic() is also
causing all realms to be queried.

    public void bootstrapPeriodic(final KeycloakSessionFactory
sessionFactory, final TimerProvider timer) {
        KeycloakModelUtils.runJobInTransaction(sessionFactory, new
KeycloakSessionTask() {

            @Override
            public void run(KeycloakSession session) {
                List<RealmModel> realms = session.realms().getRealms();
                for (final RealmModel realm : realms) {
                    List<UserStorageProviderModel> providers =
realm.getUserStorageProviders();
                    for (final UserStorageProviderModel provider :
providers) {

I'm thinking of querying only realms with user storage providers to improve
performance.

I can create a PR for this. Let me know if it's okay or if there's a better
solution than the ones I proposed.

Thanks,
Gideon

On Fri, Oct 5, 2018 at 6:22 AM Stian Thorgersen <sthorger at redhat.com> wrote:

> Keycloak simply doesn't scale well with regards to large number of realms
> today and it's not something we currently support.
>
> That's just one of several issues around large number of realms that have
> to be resolved. Another example is upgrading the server with 1700 realms is
> also going to be painful.
>
> At the moment we are not able to priorities this though. We are planning
> to resolve it, but it will be quite some time until we do.
>
> For the particular issue you've mentioned the work-around is to remove the
> realm roles from the admin composite in master realm. That will work, but
> you will only be able to login and manage realms individually.
>
> On Thu, 4 Oct 2018 at 18:07, Gideon Caranzo <gideonray at gmail.com> wrote:
>
>> Hi,
>>
>> I'm encountering slow api calls after reaching 1700 realms. I profiled it
>> and found that role checking is causing the issue particularly
>> *KeycloakModelUtils.searchFor(RoleModel
>> role, RoleModel composite, Set<String> visited)*.
>>
>> I'm using a user with "admin" role to call get realm API. And since i have
>> 1700 realms, "admin" role now have about 30K composite roles under it. The
>> line below from KeycloakModelUtils.searchFor() will load all 30K composite
>> roles causing the slow down.
>>
>>         *Set<RoleModel> compositeRoles = composite.getComposites();*
>>
>> Is there a way to avoid this issue? Or is it possible to fix the code such
>> that it will do a database query instead of searching in memory to check
>> if
>> the role exist?
>>
>> Best regards,
>> Gideon
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>