We use Jose4J[0] to create the keys and then jq[1] to modify the realm file.
See the first line of code here for a super simple example of how to generate realm
keys:
PS - this may be doable with Keycloak but Jose4J is very lightweight for writing a simple
script on a CI server.
[0]:
Scott Rossillo
Smartling | Senior Software Engineer
srossillo(a)smartling.com
On May 21, 2016, at 10:20 PM, Anthony Fryer
<anthony.fryer(a)gmail.com> wrote:
Hi Scott,
How do you generate the realm keys when creating the new keycloak dev instances? Do you
use a keycloak api or some other way? I'm interested in having a standard realm
template that is used to create new realms but would need to change the realm keys when
importing this template into keycloak.
Cheers,
Anthony
On Sat, May 21, 2016 at 3:43 AM, Scott Rossillo <srossillo(a)smartling.com
<mailto:srossillo@smartling.com>> wrote:
We’re using Keycloak on production, stage/QA, development environments and every
developer’s workstation / laptop.
While there will always be differing options on how to successfully do change management,
we’ve found a very effective method for handling Keycloak provisioning in all environments
so that developers don’t need to mess around with. We’re a continuous integration /
deployment shop using micro services and everything has to “just work” … I’ll give an
overview of our process here but please keep in mind a few things:
1. This approach works for us, I’m not saying it’s the best way
2. We do _not_ allow production config changes to be automated due to security
implications
3. We're very opinionated in our approach to configuration management and we don’t
ever modify 3rd party software databases directly. We always use APIs.
We deploy Keycloak to all environments using Docker images. On developer workstations we
use Docker Compose to orchestrate bringing up all services a developer may need, including
Keycloak.
We have 4 docker images for Keycloak:
- Keycloak Base
\- Keycloak HA
\- Keycloak Dev
- Keycloak config manager*
The base image includes all customizations necessary to bring up a Keycloak instance
configured with our modules and themes installed.
The HA instance builds off base and configures Keycloak to run as a cluster node. This is
used on stage and prod.
The dev instance builds off base and includes our realm file. On startup, this instance
loads our realm configuration if it’s not already loaded.
All docker images are built and published by the CI server and Keycloak HA can be
deployed to stage and prod after a clean CI build.
Developers are free to add clients for testing, do whatever they want, etc. to their
running dev instance. If they want to get back to our stock build, they pull the latest
Docker image from our private Docker repo and restart it.
Adding clients to stage and prod requires approval and is done by a hand. This is for
security reasons. Once a configuration change is detected on stage - say a client is added
- our CI server exports the realm from stage, changes the realm keys, and creates a new
Keycloak Dev instance with the updated realm file.
*A word about configuration management:
Obviously, the realm file we generate knows the URLs of staging services, not local or
development environment URLs. To overcome this we introduced another Docker based service
called the Keycloak configuration manger. It runs on development environments and
workstations. It’s responsible for discovering running services and updating Keycloak via
its admin endpoints to reflect the proper configuration for the given environment.
That’s it. The whole process is automated with the exception of configuration changes to
stage and prod which require a security review.
Hope this helps. Let me know if you’d like me to elaborate on anything.
Best,
Scott
Scott Rossillo
Smartling | Senior Software Engineer
srossillo(a)smartling.com <mailto:srossillo@smartling.com>
> On May 20, 2016, at 1:46 AM, Stian Thorgersen <sthorger(a)redhat.com
<mailto:sthorger@redhat.com>> wrote:
>
> Firstly, just wanted to highlight that core Keycloak team are devs, not sysadmins/ops
guys, so we have limited experience in continuous delivery and maintenance of real
production systems. Hence, we'd love input from the community on this.
>
> As it stands we don't really have a proper solution. I believe the best you can
do at the moment is either using import feature, partial import or admin rest endpoints.
Import is not going to work IMO as it requires re-creating the whole realm. Partial import
may work, but would work best for new resources rather than modifying existing resources
as it does a delete/create operation rather than attempt to modify. With the admin rest
endpoints you'd get the best control of what's going on, but obviously that leaves
a fair amount of the work.
>
> In the future we have an idea of introducing an "import directory" it would
be possible to drop json files in here that would add, modify or delete resources (realms,
clients, roles, users, whatever). This would allow dropping json files before the server
starts and the server would then import on startup. It would also be possible to do this
at runtime and new files would be detected at runtime. Finally, we also had an idea of an
offline mode to run import of this (it would basically start the server without http
listener, import files, then stop, so it could be used in a script/tool). Import is
probably not the best name for it, as it would support modify and delete as well as
"importing" new things.
>
> On 19 May 2016 at 19:53, Jesse Chahal <jessec(a)stytch.com
<mailto:jessec@stytch.com>> wrote:
> Following some of the best practices for continuous Integration and
> continuous delivery there needs to be environments for build, test,
> and production. This would mean that following these practices would
> require you to have multiple versions of keycloak at different stages
> of development cycle. Some of these environments might not have
> important persistent data while others might. In order to have builds
> transition from one environment to another there may be configuration
> changes required for a build to be valid. This is especially true when
> new services (openid clients) are being added or "default" accounts.
> I'm trying to come up with a scripted way of updating keycloak
> instances that are backed up by an RDMS. This may include adding new
> clients, adding new users, updating realm config, etc... Originally I
> was planning on simply exporting the realm config and importing it
> every time keycloak starts. If I enabled the OVERWRITE option I might
> overwrite things that I do not want overridden. This is especially
> true if there is some config that differ's based on whether it is a
> build, test, or production instance. If I don't enable it then it is
> only useful for new/blank keycloak environments. I considered using
> liquibase but since I do not have control of schema changes created by
> the keycloak team I might run into issues with my liquibase file not
> being valid after a migration/liquibase update by the keycloak team as
> my liquibase file would run after keycloak's does. There might also be
> some other unknown issues our liquibase changes conflicting somehow
> with keycloak's liquibase changes. I've also considered writing my own
> updater tool using a scripting language (python/ruby) that calls
> keycloak's rest api. The issues with this mechanism is it feels like I
> am recreating the wheel as well as not being able to find good
> documentation on keycloak's openid endpoints/url's used for different
> oauth2 flows. Even if I did find this documentation it would also
> require me to find a good openid client for the scripting language.
> This doesn't matter for our normal clients as they simply use the
> keycloak subsystems and adapters instead. I've also looked at commonly
> used server configuration software such as chef, puppet, and ansible.
> I don't see a good solution using any of those tools yet either. What
> have other people done for cases like this? Please don't tell me there
> is someone who is doing this all manually because that doesn't work in
> modern software development.
>
> - doesn't accidentally delete users
> - doesn't accidentally delete clients
> - doesn't invalidate sessions (optional)
> - works to bring up new, correctly configured, keycloak instances
> - handles applying updates to existing keycloak instances
> - can handle minor differences between keycloak instances (build,
> test, production) when updating
> - preferably can work well in rolling deployment scenario's.
> -- I hope the keycloak team is taking these into consideration when
> doing database migration between 1-2 releases. It would be nice if
> they set some specific rules for rolling updates between versions (aka
> backwards breaking changes)
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org <mailto:keycloak-user@lists.jboss.org>
>
https://lists.jboss.org/mailman/listinfo/keycloak-user
<
https://lists.jboss.org/mailman/listinfo/keycloak-user>
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org <mailto:keycloak-user@lists.jboss.org>
>
https://lists.jboss.org/mailman/listinfo/keycloak-user
<
https://lists.jboss.org/mailman/listinfo/keycloak-user>
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org <mailto:keycloak-user@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-user
<
https://lists.jboss.org/mailman/listinfo/keycloak-user>