[keycloak-user] Performance issues with large number of realms (500+)

[Chris Savory]

Stain,

I completely understand your position regarding multi-tenancy and Keycloak; that you would like the community to contribute back on identifying issues and improving the performance for 10+ realms configuration.  

Out of curiosity, does the same apply to Redhat SSO customers who are already paying for support? 

--
Christopher Savory
 
 <http://www.edlogics.com/>

On 10/19/16, 7:41 AM, "Stian Thorgersen" <sthorger at redhat.com> wrote:

    I can appreciate that the multi-tenancy concept does conflict with what I'm
    saying here, but that was contributed by community and in that use-case
    they only had a few realms. We've been asked this questions a few times on
    the mailing list though and we have responded with it may work, but we
    haven't tested it, nor have we planned for it to be used that way.
    
    I'm not saying it'll be a problem with more than 10, just saying that's
    what we had in mind as what the maximum people would have. Unless of course
    you're building out a SaaS or something on top of Keycloak. In that case I
    would believe you're probably making money out of what you are doing and
    you should be able to afford to spend some time contributing to Keycloak.
    
    We simply don't have the resources to test this scenario and improve the
    performance around it.
    
    On 19 October 2016 at 14:04, Gabriel Lavoie <glavoie at gmail.com> wrote:
    
    > Hi Stian,
    >      your answer does surprise me (and my team) a bit as the Keycloak
    > usage examples, documentation, some blog post points to multi-tenancy with
    > the realm concept. We did not find any documentation that would discourage
    > it. I was curious about possible guidelines and 10 realms seems to be a
    > very low number.
    >
    > This said, we took time almost a year ago to evaluate the architectural
    > limits (custom themes/providers are not an issue) and did load tests (with
    > large number of users and realms) on the different authentication endpoints
    > we needed to use and no big issue appeared at that time. We did expect to
    > find issues when ramping up the usage in QA/Production.
    >
    > Keycloak is secured internally in our SaaS environment (only the necessary
    > paths/realms are exposed) and we automate the whole realm
    > creation/management to effectively manage security concerns. All we are
    > doing is creating a realm with a very specific configuration and repeating
    > the operation N times.
    >
    > For the two issues I found, I already have ideas of how to fix them (and 1
    > fix proposal almost ready). In the meantime, we will consider different
    > deployment/provisioning options as workarounds.
    >
    > Regards,
    >
    > Gabriel
    >
    > 2016-10-18 10:04 GMT-04:00 Stian Thorgersen <sthorger at redhat.com>:
    >
    >> Keycloak was not designed to support multi-tenancy directly. We made the
    >> decision early on that we can't support true multi-tenancy and that has to
    >> be done through separate instances. This is for security reasons as well as
    >> the fact that we can't sandbox everything (like custom providers, custom
    >> themes, etc.).
    >>
    >> In that regards we have never tested with high amounts of realms as we
    >> expect there to be few realms (up to 10 most likely). Nor will we test
    >> this. We won't fix any issues related to high number of realms for this
    >> reason either.
    >>
    >> I'm not saying that we don't appreciate your case, but we have other
    >> priorities that we need to work on.
    >>
    >> However, if you are able to provide PRs that do not have any side effects
    >> (and also doesn't significantly complicate things) we would be happy to
    >> accept them.
    >>
    >> On 18 October 2016 at 15:49, Gabriel Lavoie <glavoie at gmail.com> wrote:
    >>
    >>> Hi,
    >>>      our Keycloak setup is being used in a multi-tenant fashion with a
    >>> large number of realms assigned to different instances of our application
    >>> (multiple customers). We are now seeing a few performance issues with the
    >>> startup and administration.
    >>>
    >>> First question: Do you have guidelines on a maximum number of realms that
    >>> Keycloak should support before we split in smaller clusters?
    >>>
    >>> I traced at least 2 things in the KC code that could be improved. Should
    >>> I
    >>> open tickets for both?
    >>>
    >>> 1 - Slow startup (5 minutes with 500 realms):
    >>> In the KeycloakApplication class constructor, the "isNewInstall()" test
    >>> to
    >>> check if the master realm must be created triggers the loading and
    >>> caching
    >>> of all realms. This loading seems to be hit with a similar issue that I
    >>> had
    >>> in the past with realm export: https://issues.jboss.org/brows
    >>> e/KEYCLOAK-2413
    >>>
    >>> The named query that gets executed a lot of times in
    >>> RealmAdapter.getAuthenticationExecution() triggers a flush within
    >>> Hibernate
    >>> every times. If the flush mode gets set to "COMMIT" (can't be changed by
    >>> default but I tested it), the loading time goes down to approximately 30
    >>> secs which is acceptable.
    >>>
    >>> It would likely be a good idea to create a read-only transaction with the
    >>> flush mode set to COMMIT during startup to pre-fill the cache, then
    >>> continue with the rest of the initialization. When the cache is filled,
    >>> accessing info on all realms seems to be fine.
    >>>
    >>> 2 - Slow display of the master realm admin screen.
    >>> When accessing the admin screen, AdminConsole.whoAmI() eventually process
    >>> all the roles on all the realm for the admin
    >>> user. KeycloakModelUtils.searchFor() gets called a lot of times to
    >>> navigate
    >>> through all the composite permissions. With 500 realms, the user has
    >>> about
    >>> 6500 total permissions available. This part of the code would likely
    >>> benefit a lot from a cache of the exploded composite permissions.
    >>>
    >>> Thanks,
    >>>
    >>> Gabriel
    >>>
    >>> --
    >>> Gabriel Lavoie
    >>> glavoie at gmail.com
    >>> _______________________________________________
    >>> keycloak-user mailing list
    >>> keycloak-user at lists.jboss.org
    >>> https://lists.jboss.org/mailman/listinfo/keycloak-user
    >>>
    >>
    >>
    >
    >
    > --
    > Gabriel Lavoie
    > glavoie at gmail.com
    >
    _______________________________________________
    keycloak-user mailing list
    keycloak-user at lists.jboss.org
    https://lists.jboss.org/mailman/listinfo/keycloak-user