[keycloak-user] How to apply updates to keycloak instances

Mon May 23 16:49:33 EDT 2016

Thanks, I'll check it out.

On 05:38, Tue, 24/05/2016 Scott Rossillo <srossillo at smartling.com> wrote:

> We use Jose4J[0] to create the keys and then jq[1] to modify the realm
> file.
>
>  See the first line of code here for a super simple example of how to
> generate realm keys:
> https://bitbucket.org/b_c/jose4j/wiki/JWT%20Examples
>
> PS - this may be doable with Keycloak but Jose4J is very lightweight for
> writing a simple script on a CI server.
>
> [0]: https://bitbucket.org/b_c/jose4j
> [1]: https://stedolan.github.io/jq/
>
>
> Scott Rossillo
> Smartling | Senior Software Engineer
> srossillo at smartling.com
>
> On May 21, 2016, at 10:20 PM, Anthony Fryer <anthony.fryer at gmail.com>
> wrote:
>
> Hi Scott,
>
> How do you generate the realm keys when creating the new keycloak dev
> instances?  Do you use a keycloak api or some other way?  I'm interested in
> having a standard realm template that is used to create new realms but
> would need to change the realm keys when importing this template into
> keycloak.
>
> Cheers,
>
> Anthony
>
> On Sat, May 21, 2016 at 3:43 AM, Scott Rossillo <srossillo at smartling.com>
> wrote:
>
>> We’re using Keycloak on production, stage/QA, development environments
>> and every developer’s workstation / laptop.
>>
>> While there will always be differing options on how to successfully do
>> change management, we’ve found a very effective method for handling
>> Keycloak provisioning in all environments so that developers don’t need to
>> mess around with. We’re a continuous integration / deployment shop using
>> micro services and everything has to “just work” … I’ll give an overview of
>> our process here but please keep in mind a few things:
>>
>> 1. This approach works for us, I’m not saying it’s the best way
>> 2. We do _not_ allow production config changes to be automated due to
>> security implications
>> 3. We're very opinionated in our approach to configuration management and
>> we don’t ever modify 3rd party software databases directly. We always use
>> APIs.
>>
>> We deploy Keycloak to all environments using Docker images. On developer
>> workstations we use Docker Compose to orchestrate bringing up all services
>> a developer may need, including Keycloak.
>>
>> We have 4 docker images for Keycloak:
>> - Keycloak Base
>>    \- Keycloak HA
>>    \- Keycloak Dev
>> - Keycloak config manager*
>>
>> The base image includes all customizations necessary to bring up a
>> Keycloak instance configured with our modules and themes installed.
>> The HA instance builds off base and configures Keycloak to run as a
>> cluster node. This is used on stage and prod.
>> The dev instance builds off base and includes our realm file. On startup,
>> this instance loads our realm configuration if it’s not already loaded.
>>
>> All docker images are built and published by the CI server and Keycloak
>> HA can be deployed to stage and prod after a clean CI build.
>>
>> Developers are free to add clients for testing, do whatever they want,
>> etc. to their running dev instance. If they want to get back to our stock
>> build, they pull the latest Docker image from our private Docker repo and
>> restart it.
>>
>> Adding clients to stage and prod requires approval and is done by a hand.
>> This is for security reasons. Once a configuration change is detected on
>> stage - say a client is added - our CI server exports the realm from stage,
>> changes the realm keys, and creates a new Keycloak Dev instance with the
>> updated realm file.
>>
>> *A word about configuration management:
>>
>> Obviously, the realm file we generate knows the URLs of staging services,
>> not local or development environment URLs. To overcome this we introduced
>> another Docker based service called the Keycloak configuration manger. It
>> runs on development environments and workstations. It’s responsible for
>> discovering running services and updating Keycloak via its admin endpoints
>> to reflect the proper configuration for the given environment.
>>
>> That’s it. The whole process is automated with the exception of
>> configuration changes to stage and prod which require a security review.
>>
>> Hope this helps. Let me know if you’d like me to elaborate on anything.
>>
>> Best,
>> Scott
>>
>> Scott Rossillo
>> Smartling | Senior Software Engineer
>> srossillo at smartling.com
>>
>> On May 20, 2016, at 1:46 AM, Stian Thorgersen <sthorger at redhat.com>
>> wrote:
>>
>> Firstly, just wanted to highlight that core Keycloak team are devs, not
>> sysadmins/ops guys, so we have limited experience in continuous delivery
>> and maintenance of real production systems. Hence, we'd love input from the
>> community on this.
>>
>> As it stands we don't really have a proper solution. I believe the best
>> you can do at the moment is either using import feature, partial import or
>> admin rest endpoints. Import is not going to work IMO as it requires
>> re-creating the whole realm. Partial import may work, but would work best
>> for new resources rather than modifying existing resources as it does a
>> delete/create operation rather than attempt to modify. With the admin rest
>> endpoints you'd get the best control of what's going on, but obviously that
>> leaves a fair amount of the work.
>>
>> In the future we have an idea of introducing an "import directory" it
>> would be possible to drop json files in here that would add, modify or
>> delete resources (realms, clients, roles, users, whatever). This would
>> allow dropping json files before the server starts and the server would
>> then import on startup. It would also be possible to do this at runtime and
>> new files would be detected at runtime. Finally, we also had an idea of an
>> offline mode to run import of this (it would basically start the server
>> without http listener, import files, then stop, so it could be used in a
>> script/tool). Import is probably not the best name for it, as it would
>> support modify and delete as well as "importing" new things.
>>
>> On 19 May 2016 at 19:53, Jesse Chahal <jessec at stytch.com> wrote:
>>
>>> Following some of the best practices for continuous Integration and
>>> continuous delivery there needs to be environments for build, test,
>>> and production. This would mean that following these practices would
>>> require you to have multiple versions of keycloak at different stages
>>> of development cycle. Some of these environments might not have
>>> important persistent data while others might. In order to have builds
>>> transition from one environment to another there may be configuration
>>> changes required for a build to be valid. This is especially true when
>>> new services (openid clients) are being added or "default" accounts.
>>> I'm trying to come up with a scripted way of updating keycloak
>>> instances that are backed up by an RDMS. This may include adding new
>>> clients, adding new users, updating realm config, etc... Originally I
>>> was planning on simply exporting the realm config and importing it
>>> every time keycloak starts. If I enabled the OVERWRITE option I might
>>> overwrite things that I do not want overridden. This is especially
>>> true if there is some config that differ's based on whether it is a
>>> build, test, or production instance. If I don't enable it then it is
>>> only useful for new/blank keycloak environments. I considered using
>>> liquibase but since I do not have control of schema changes created by
>>> the keycloak team I might run into issues with my liquibase file not
>>> being valid after a migration/liquibase update by the keycloak team as
>>> my liquibase file would run after keycloak's does. There might also be
>>> some other unknown issues our liquibase changes conflicting somehow
>>> with keycloak's liquibase changes. I've also considered writing my own
>>> updater tool using a scripting language (python/ruby) that calls
>>> keycloak's rest api. The issues with this mechanism is it feels like I
>>> am recreating the wheel as well as not being able to find good
>>> documentation on keycloak's openid endpoints/url's used for different
>>> oauth2 flows. Even if I did find this documentation it would also
>>> require me to find a good openid client for the scripting language.
>>> This doesn't matter for our normal clients as they simply use the
>>> keycloak subsystems and adapters instead. I've also looked at commonly
>>> used server configuration software such as chef, puppet, and ansible.
>>> I don't see a good solution using any of those tools yet either. What
>>> have other people done for cases like this? Please don't tell me there
>>> is someone who is doing this all manually because that doesn't work in
>>> modern software development.
>>>
>>> - doesn't accidentally delete users
>>> - doesn't accidentally delete clients
>>> - doesn't invalidate sessions (optional)
>>> - works to bring up new, correctly configured, keycloak instances
>>> - handles applying updates to existing keycloak instances
>>> - can handle minor differences between keycloak instances (build,
>>> test, production) when updating
>>> - preferably can work well in rolling deployment scenario's.
>>> -- I hope the keycloak team is taking these into consideration when
>>> doing database migration between 1-2 releases. It would be nice if
>>> they set some specific rules for rolling updates between versions (aka
>>> backwards breaking changes)
>>> _______________________________________________
>>> keycloak-user mailing list
>>> keycloak-user at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>>
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>>
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160523/bad923f3/attachment.html 