4:40 p.m.
Hello,
Our organization has been using Keycloak over the last few years. During this time,
several versions and implementation approaches of Keycloak have popped up in the
organization as various organizational units leveraged Keycloak.
We are now at the point of taking Keycloak to the next level of maturity within the
organization with a common architecture and governance model around Keycloak/IDAM.
We have convened a working group to take our experiences to-date and formulate an
architecture which the organization can move forward with. The major point of contention
with the future architecture is the nature in which the instances and realms are
deployed.
To this end, I am looking for some feedback from the community regarding the most scalable
architectural blueprint/recommendation to help achieve the following requirements and
questions:
Here is a list of our assumptions/constraints:
1. The organization consists of 10 organizational units (i.e. realms).
2. Each organization unit supports 10-15 applications (i.e. clients) requiring
authentication/authorization.
3. The primary application profile is a web application. (i.e. keycloak access type of
'confidential')
4. The organization is starting to developing an increasing number of web services which
leverage bearer-only authn/authz.
5. For the organization, Keycloak would support 100,000 users.
6. Of the 100,000 users, 1-2% of those users would be federated via Active Directory.
7. Within an organization unit, users should be able to leverage SSO for any application
within the organizational unit.
8. The primary usage of applications are between core business hours.
9. The applications are accessible 24x7.
10. On any given day, about 20% of the total user base may log into at least one
application.
11. Due to inactivity requirements, users may typically have to re-authenticate multiple
times during the day.
12. The organization has a desire to maintain a common set of IDAM policies and reporting
(i.e. governance) across all organizational units.
13. The organization would provide a default template for all organizational units.
14. Each organization unit may modify/create their own template as business requirements
dictate.
15. Keycloak should be clustered for high availability.
16. Keycloak environment would be hosted on AWS, more than likely EC2 instances.
17. Client applications also hosted in AWS.
18. Keycloak's database would be PostgreSQL hosted in AWS RDS.
A few questions/concerns of the working group:
A. Is there any information available on the maximum size of an Keycloak installation?
Will Keycloak be scalable and performant given the above assumptions and constraints.
B. What's the best recommendation for distributing the Keycloak instances and realms.
Right now the group has three options on the table: 1) A single Keycloak install per
application (i.e. client); 2) A single Keycloak install per organizational unit (i.e.
realm); or 3) A single Keycloak install per organization (i.e. serving all realms and
clients).
C. A major concern the group has with a single Keycloak install (#3 in previous bullet) is
the high-availability in terms of performance and concerns of a rouge client affecting
other applications negatively. What is the community's recommendation for addressing
this concern?
D. Another major concern the group has with a single Keycloak install is the restarts that
are necessary when an organization unit deploys a new or updated template. The concern is
that all applications would be unavailable during the restart. We would be operating in
a clustered environment, is the best solution to this concern restarting individual
members of the cluster rather than the entire cluster?
E. For reporting and governance processes, the Keycloak API performs quite poorly when we
execute use cases such as "Report all Users of an Application". Given the
version we are currently on, to accomplish this we need to query all users in the realm
and then filter the users if they have the client/role combination. We understand that a
future release addresses this use case, but in the meantime the concern is such a query
will negatively affect all other clients using Keycloak. Any recommendations on handling
this use case prior to Keycloak 4.x?
F. Upgrading Versions of Keycloak. We have experienced some difficulty of upgrading
versions on server-side (we need to export, import vs a simple DB backup and deployment).
What is the recommendations for handling the upgrade of Keycloak from one version to the
next given the size of our user base?
I'm sorry for the long post, hopefully folks get to this point. Any insight that we
could receive would be greatly appreciated. We are at a critical cross-roads in our
Keycloak adoption and want to ensure we do this correctly.
Thanks!
Dave
Dave Everson
Application Development Team Lead | Environmental Health
Minnesota IT Services | Partners in Minnesota Department of Health
625 Robert Street North
St. Paul, MN 55155
O: 651-201-5146
Information Technology for Minnesota Government | mn.gov/mnit<http://mn.gov/mnit>
[Minnesota IT Services Logo]
[Facebook logo]<https://www.facebook.com/MN.ITServices>[LinkedIn
logo]<https://www.linkedin.com/company/mn-it-services>[Twitter
logo]<https://twitter.com/mnit_services>
8:06 p.m.
New subject: FW: Architectural Blueprint/Recommendations
Good Afternoon, If this is not the appropriate place to ask these types of questions,
where does the community suggest that I seek these answers? Thanks! Dave
-----Original Message-----
From: keycloak-user-bounces(a)lists.jboss.org <keycloak-user-bounces(a)lists.jboss.org>
On Behalf Of Everson, David (MNIT)
Sent: Monday, June 18, 2018 9:41 AM
To: keycloak-user(a)lists.jboss.org
Subject: [keycloak-user] Architectural Blueprint/Recommendations
Hello,
Our organization has been using Keycloak over the last few years. During this time,
several versions and implementation approaches of Keycloak have popped up in the
organization as various organizational units leveraged Keycloak.
We are now at the point of taking Keycloak to the next level of maturity within the
organization with a common architecture and governance model around Keycloak/IDAM.
We have convened a working group to take our experiences to-date and formulate an
architecture which the organization can move forward with. The major point of contention
with the future architecture is the nature in which the instances and realms are
deployed.
To this end, I am looking for some feedback from the community regarding the most scalable
architectural blueprint/recommendation to help achieve the following requirements and
questions:
Here is a list of our assumptions/constraints:
1. The organization consists of 10 organizational units (i.e. realms).
2. Each organization unit supports 10-15 applications (i.e. clients) requiring
authentication/authorization.
3. The primary application profile is a web application. (i.e. keycloak access type of
'confidential') 4. The organization is starting to developing an increasing number
of web services which leverage bearer-only authn/authz.
5. For the organization, Keycloak would support 100,000 users.
6. Of the 100,000 users, 1-2% of those users would be federated via Active Directory.
7. Within an organization unit, users should be able to leverage SSO for any application
within the organizational unit.
8. The primary usage of applications are between core business hours.
9. The applications are accessible 24x7.
10. On any given day, about 20% of the total user base may log into at least one
application.
11. Due to inactivity requirements, users may typically have to re-authenticate multiple
times during the day.
12. The organization has a desire to maintain a common set of IDAM policies and reporting
(i.e. governance) across all organizational units.
13. The organization would provide a default template for all organizational units.
14. Each organization unit may modify/create their own template as business requirements
dictate.
15. Keycloak should be clustered for high availability.
16. Keycloak environment would be hosted on AWS, more than likely EC2 instances.
17. Client applications also hosted in AWS.
18. Keycloak's database would be PostgreSQL hosted in AWS RDS.
A few questions/concerns of the working group:
A. Is there any information available on the maximum size of an Keycloak installation?
Will Keycloak be scalable and performant given the above assumptions and constraints.
B. What's the best recommendation for distributing the Keycloak instances and realms.
Right now the group has three options on the table: 1) A single Keycloak install per
application (i.e. client); 2) A single Keycloak install per organizational unit (i.e.
realm); or 3) A single Keycloak install per organization (i.e. serving all realms and
clients).
C. A major concern the group has with a single Keycloak install (#3 in previous bullet) is
the high-availability in terms of performance and concerns of a rouge client affecting
other applications negatively. What is the community's recommendation for addressing
this concern?
D. Another major concern the group has with a single Keycloak install is the restarts that
are necessary when an organization unit deploys a new or updated template. The concern is
that all applications would be unavailable during the restart. We would be operating in
a clustered environment, is the best solution to this concern restarting individual
members of the cluster rather than the entire cluster?
E. For reporting and governance processes, the Keycloak API performs quite poorly when we
execute use cases such as "Report all Users of an Application". Given the
version we are currently on, to accomplish this we need to query all users in the realm
and then filter the users if they have the client/role combination. We understand that a
future release addresses this use case, but in the meantime the concern is such a query
will negatively affect all other clients using Keycloak. Any recommendations on handling
this use case prior to Keycloak 4.x?
F. Upgrading Versions of Keycloak. We have experienced some difficulty of upgrading
versions on server-side (we need to export, import vs a simple DB backup and deployment).
What is the recommendations for handling the upgrade of Keycloak from one version to the
next given the size of our user base?
I'm sorry for the long post, hopefully folks get to this point. Any insight that we
could receive would be greatly appreciated. We are at a critical cross-roads in our
Keycloak adoption and want to ensure we do this correctly.
Thanks!
Dave
Dave Everson
Application Development Team Lead | Environmental Health Minnesota IT Services | Partners
in Minnesota Department of Health
625 Robert Street North
St. Paul, MN 55155
O: 651-201-5146
Information Technology for Minnesota Government | mn.gov/mnit<http://mn.gov/mnit>
[Minnesota IT Services Logo] [Facebook
logo]<https://www.facebook.com/MN.ITServices>[LinkedIn
logo]<https://www.linkedin.com/company/mn-it-services>[Twitter
logo]<https://twitter.com/mnit_services>
12:26 a.m.
Hi David,
Please see the answers and remarks inline.
On Mon, 2018-06-18 at 14:40 +0000, Everson, David (MNIT) wrote:
(skipped)
15. Keycloak should be clustered for high availability.
16. Keycloak environment would be hosted on AWS, more than likely EC2
instances.
17. Client applications also hosted in AWS.
18. Keycloak's database would be PostgreSQL hosted in AWS RDS.
Speaking of Keycloak on AWS, this is absolutely doable, but not that
trivial. Please be sure to have read the document [1], especially the
"Troubleshooting AWS specifics" part, and relevant ML postings [2].
Long story short, AWS doesn't allow for IP multicast between the nodes,
which is the default node discovery mode in JGroups (the backbone of
Keycloak clustering). You should use S3_PING or JDBC_PING instead.
A few questions/concerns of the working group:
A. Is there any information available on the maximum size of an
Keycloak installation? Will Keycloak be scalable and performant given
the above assumptions and constraints.
AFAIK, nobody has performed actual Keycloak benchmarking yet
(publicly). There's however a Keycloak benchmarking suite based on
Gatling [3]. It hasn't been updated for about two years, so first we'll
need to make sure it works with recent Keycloak versions.
B. What's the best recommendation for distributing the Keycloak
instances and realms. Right now the group has three options on the
table: 1) A single Keycloak install per application (i.e.
client); 2) A single Keycloak install per organizational unit (i.e.
realm); or 3) A single Keycloak install per organization (i.e.
serving all realms and clients).
The pros for A and B is obviously that you get some degree of
separation/isolation, which might be good from the security and
availability POV. However, this comes at a price of complexity; you'll
have to deploy, monitor & maintain each separate instance / group of
instances, each having different configs and dedicated database.
Another big issue is load distribution. I doubt that your
clients/realms all have equal, uniform load patterns. Given that each
Keycloak instance will have its hardware limitations (CPU, RAM), you
potentially end up with some nodes overloaded and others idle. The C
scenario is obviously free from this issue.
C. A major concern the group has with a single Keycloak install (#3
in previous bullet) is the high-availability in terms of performance
and concerns of a rouge client affecting other applications
negatively. What is the community's recommendation for addressing
this concern?
As you will necessarily have a load balancer / reverse proxy in front
of your Keycloak cluster, you can enforce rate limiting / throttling on
your load balancer. For example, haproxy implements rate limiting based
on IP addresses, URLs and HTTP headers [4].
D. Another major concern the group has with a single Keycloak install
is the restarts that are necessary when an organization unit deploys
a new or updated template. The concern is that all applications
would be unavailable during the restart. We would be operating in a
clustered environment, is the best solution to this concern
restarting individual members of the cluster rather than the entire
cluster?
Could you please elaborate on template deployment? In Keycloak
parlance, "templates" can be understood either as "client templates"
or
"HTML templates" (within custom GUI themes).
Client templates surely can be created/updated via GUI or REST API,
without the need for restart. For GUI themes, they can be deployed a)
as Wildfly modules, b) via "themes" directory. While the former option
indeed requires restart, the latter does not. Keycloak 4.x also adds c)
hot deployment of themes by dropping theme JARs into the "deployments"
directory.
E. For reporting and governance processes, the Keycloak API performs
quite poorly when we execute use cases such as "Report all Users of
an Application". Given the version we are currently on, to
accomplish this we need to query all users in the realm and then
filter the users if they have the client/role combination. We
understand that a future release addresses this use case, but in the
meantime the concern is such a query will negatively affect all other
clients using Keycloak. Any recommendations on handling this use
case prior to Keycloak 4.x?
Is this indeed addressed by Keycloak 4.x? (just wondering, couldn't
find any info)
Keycloak admin REST API has an endpoint called "Return List of Users
that have the specified role name", see [5] (identical for KC 3.x and
4.x). You could use this endpoint, however you will have to iterate
over client roles and then merge and de-duplicate the results. Anyway,
this should be much more efficient than your current approach.
In general, this looks like a classical use case for Realm Resource
Provider [6]. The query you described easily maps to a single SQL/JPQL
statement, so you could implement a custom REST resource that would
execute exactly that query and return results.
Unfortunately, custom REST resources in Keycloak are public by default
(protected resources should become a part of the hypothetical Admin
Resource SPI somewhere in the future). However, you can implement that
(relatively) easily with the techniques demonstrated in Beercloak [7].
F. Upgrading Versions of Keycloak. We have experienced some
difficulty of upgrading versions on server-side (we need to export,
import vs a simple DB backup and deployment). What is the
recommendations for handling the upgrade of Keycloak from one version
to the next given the size of our user base?
Could you please elaborate a bit on the problems that you're facing?
The export/import scenario is relevant for database upgrades (e.g.
PostgreSQL 9 -> 10), but Keycloak does ship migration scripts that
should upgrade the data+metadata automatically. Why doesn't that work
in your case? Let us know, probably this could be fixed.
I'm sorry for the long post, hopefully folks get to this point. Any
insight that we could receive would be greatly appreciated. We are at
a critical cross-roads in our Keycloak adoption and want to ensure we
do this correctly.
Sorry it took so long to reply. Keycloak is a great product, I hope it
fulfills your needs. Good luck!
Dmitry Telegin
CTO, Acutus s.r.o.
Keycloak Consulting and Training
[1] https://blog.keycloak.org/2018/01/keycloak-cross-data-center-setup-
in-aws.html
[2] https://www.keycloak.org/search.html?q=aws
[3] https://github.com/rvansa/keycloak-benchmark
[4] https://blog.codecentric.de/en/2014/12/haproxy-http-header-rate-lim
iting/
[5] https://www.keycloak.org/docs-api/3.4/rest-api/index.html
[6] https://www.keycloak.org/docs/latest/server_development/index.html#
_extensions_rest
[7] https://github.com/dteleguin/beercloak
Thanks!
Dave
Dave Everson
Application Development Team Lead | Environmental Health
Minnesota IT Services | Partners in Minnesota Department of Health
625 Robert Street North
St. Paul, MN 55155
O: 651-201-5146
Information Technology for Minnesota Government |
mn.gov/mnit<http://mn.gov/mnit>
[Minnesota IT Services Logo]
[Facebook logo]<https://www.facebook.com/MN.ITServices>[LinkedIn
logo]<https://www.linkedin.com/company/mn-it-services>[Twitter
logo]<https://twitter.com/mnit_services>
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user
2409
days inactive
2412
days old
2 comments
2 participants
participants (2)
-
Dmitry Telegin -
Everson, David (MNIT)