[keycloak-user] How to apply updates to keycloak instances

Fri May 20 13:43:07 EDT 2016

We’re using Keycloak on production, stage/QA, development environments and every developer’s workstation / laptop.

While there will always be differing options on how to successfully do change management, we’ve found a very effective method for handling Keycloak provisioning in all environments so that developers don’t need to mess around with. We’re a continuous integration / deployment shop using micro services and everything has to “just work” … I’ll give an overview of our process here but please keep in mind a few things:

1. This approach works for us, I’m not saying it’s the best way
2. We do _not_ allow production config changes to be automated due to security implications 
3. We're very opinionated in our approach to configuration management and we don’t ever modify 3rd party software databases directly. We always use APIs.

We deploy Keycloak to all environments using Docker images. On developer workstations we use Docker Compose to orchestrate bringing up all services a developer may need, including Keycloak.

We have 4 docker images for Keycloak:
- Keycloak Base
   \- Keycloak HA
   \- Keycloak Dev
- Keycloak config manager*

The base image includes all customizations necessary to bring up a Keycloak instance configured with our modules and themes installed.
The HA instance builds off base and configures Keycloak to run as a cluster node. This is used on stage and prod.
The dev instance builds off base and includes our realm file. On startup, this instance loads our realm configuration if it’s not already loaded.

All docker images are built and published by the CI server and Keycloak HA can be deployed to stage and prod after a clean CI build.

Developers are free to add clients for testing, do whatever they want, etc. to their running dev instance. If they want to get back to our stock build, they pull the latest Docker image from our private Docker repo and restart it.

Adding clients to stage and prod requires approval and is done by a hand. This is for security reasons. Once a configuration change is detected on stage - say a client is added - our CI server exports the realm from stage, changes the realm keys, and creates a new Keycloak Dev instance with the updated realm file.

*A word about configuration management:

Obviously, the realm file we generate knows the URLs of staging services, not local or development environment URLs. To overcome this we introduced another Docker based service called the Keycloak configuration manger. It runs on development environments and workstations. It’s responsible for discovering running services and updating Keycloak via its admin endpoints to reflect the proper configuration for the given environment.

That’s it. The whole process is automated with the exception of configuration changes to stage and prod which require a security review.

Hope this helps. Let me know if you’d like me to elaborate on anything.

Best,
Scott

Scott Rossillo
Smartling | Senior Software Engineer
srossillo at smartling.com

> On May 20, 2016, at 1:46 AM, Stian Thorgersen <sthorger at redhat.com> wrote:
> 
> Firstly, just wanted to highlight that core Keycloak team are devs, not sysadmins/ops guys, so we have limited experience in continuous delivery and maintenance of real production systems. Hence, we'd love input from the community on this.
> 
> As it stands we don't really have a proper solution. I believe the best you can do at the moment is either using import feature, partial import or admin rest endpoints. Import is not going to work IMO as it requires re-creating the whole realm. Partial import may work, but would work best for new resources rather than modifying existing resources as it does a delete/create operation rather than attempt to modify. With the admin rest endpoints you'd get the best control of what's going on, but obviously that leaves a fair amount of the work.
> 
> In the future we have an idea of introducing an "import directory" it would be possible to drop json files in here that would add, modify or delete resources (realms, clients, roles, users, whatever). This would allow dropping json files before the server starts and the server would then import on startup. It would also be possible to do this at runtime and new files would be detected at runtime. Finally, we also had an idea of an offline mode to run import of this (it would basically start the server without http listener, import files, then stop, so it could be used in a script/tool). Import is probably not the best name for it, as it would support modify and delete as well as "importing" new things.
> 
> On 19 May 2016 at 19:53, Jesse Chahal <jessec at stytch.com <mailto:jessec at stytch.com>> wrote:
> Following some of the best practices for continuous Integration and
> continuous delivery there needs to be environments for build, test,
> and production. This would mean that following these practices would
> require you to have multiple versions of keycloak at different stages
> of development cycle. Some of these environments might not have
> important persistent data while others might. In order to have builds
> transition from one environment to another there may be configuration
> changes required for a build to be valid. This is especially true when
> new services (openid clients) are being added or "default" accounts.
> I'm trying to come up with a scripted way of updating keycloak
> instances that are backed up by an RDMS. This may include adding new
> clients, adding new users, updating realm config, etc... Originally I
> was planning on simply exporting the realm config and importing it
> every time keycloak starts. If I enabled the OVERWRITE option I might
> overwrite things that I do not want overridden. This is especially
> true if there is some config that differ's based on whether it is a
> build, test, or production instance. If I don't enable it then it is
> only useful for new/blank keycloak environments. I considered using
> liquibase but since I do not have control of schema changes created by
> the keycloak team I might run into issues with my liquibase file not
> being valid after a migration/liquibase update by the keycloak team as
> my liquibase file would run after keycloak's does. There might also be
> some other unknown issues our liquibase changes conflicting somehow
> with keycloak's liquibase changes. I've also considered writing my own
> updater tool using a scripting language (python/ruby) that calls
> keycloak's rest api. The issues with this mechanism is it feels like I
> am recreating the wheel as well as not being able to find good
> documentation on keycloak's openid endpoints/url's used for different
> oauth2 flows. Even if I did find this documentation it would also
> require me to find a good openid client for the scripting language.
> This doesn't matter for our normal clients as they simply use the
> keycloak subsystems and adapters instead. I've also looked at commonly
> used server configuration software such as chef, puppet, and ansible.
> I don't see a good solution using any of those tools yet either. What
> have other people done for cases like this? Please don't tell me there
> is someone who is doing this all manually because that doesn't work in
> modern software development.
> 
> - doesn't accidentally delete users
> - doesn't accidentally delete clients
> - doesn't invalidate sessions (optional)
> - works to bring up new, correctly configured, keycloak instances
> - handles applying updates to existing keycloak instances
> - can handle minor differences between keycloak instances (build,
> test, production) when updating
> - preferably can work well in rolling deployment scenario's.
> -- I hope the keycloak team is taking these into consideration when
> doing database migration between 1-2 releases. It would be nice if
> they set some specific rules for rolling updates between versions (aka
> backwards breaking changes)
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org <mailto:keycloak-user at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-user <https://lists.jboss.org/mailman/listinfo/keycloak-user>
> 
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160520/a2618e24/attachment-0001.html 