[keycloak-user] How to apply updates to keycloak instances

Thu May 26 13:41:04 EDT 2016

On 26 May 2016 at 19:11, Scott Rossillo <srossillo at smartling.com> wrote:

> I guess it’s a matter of requirements, but with micro service
> architectures there’s usually some sort of discovery mechanism required to
> locale services at runtime. Netflix offers Eureka and then there’s etcd
> from CoreOS that’s being used by Kubernetes. My point is that even if
> Keycloak devs build some sort of way of picking up changes from the
> filesystem on startup, that doesn’t solve all use cases.
>

The problem is continuous integration right, and pushing changes from a
test environment into production? So you need a reliable way to apply
changes to both environments.

>
> It doesn’t solve issues with Dockerized deployments and it doesn’t solve
> the issue where services have to be located at runtime
>

What are the issues it doesn't solve?

>
> Scott Rossillo
> Smartling | Senior Software Engineer
> srossillo at smartling.com
>
> On May 26, 2016, at 2:27 AM, Stian Thorgersen <sthorger at redhat.com> wrote:
>
>
>
> On 26 May 2016 at 02:15, Jesse Chahal <jessec at stytch.com> wrote:
>
>> @Stian
>> The approach described sounds similar to liquibase to me but with json
>> and specific to keycloak. I feel like a lot of possible bugs could
>> arise from this approach or at least quite a few feature requests.
>> Would each json file only contain a single change? Would order matter
>> in how they get applied if you put a bunch of json files in this
>> directory at once? Can the same file be applied multiple times? These
>> are the kind of issues I would expect to come up with this type of
>> change management system. When I mentioned write our own tool/script
>> to do it I was kind of thinking of a writing a liquibase like system
>> that calls keycloak's rest api.
>>
>
> We haven't figured out all the details, but what you are proposing sounds
> better. A single document that lists all changes, that can also import
> other files, sorts out the ordering and we could add same type of ids as
> Liquibase does to changesets.
>
> You could write it to use the rest api, then use a separate db to store
> what changes have been applied, but would be better if Keycloak deals with
> loading the changes directly as it can write to the db what changes have
> been applied.
>
> One big issue is what happens if manual changes are done through the admin
> console. One though (although probably very tricky to get right) is that
> changes done through the admin console is added to the changeset.
>
>
>>
>> @ Scott
>> If I would compare the solution you mentioned to one of the options I
>> listed in my original question  "I've also considered writing my own
>> updater tool using a scripting language (python/ruby) that calls
>> keycloak's rest api." The worrying thing to me is that there is
>> another piece of code that needs to maintained by our company and
>> requires quite a bit of knowledge of keycloak's rest api. There would
>> probably need to be some serious thought put into the architecture of
>> the tool as well. Without a doubt it does provide the most control. We
>> also live by a different methodology in regards to updating production
>> clusters. From our perspective it is more of an issue to update
>> manually as it becomes much easier to miss a step or in someway screw
>> up if steps are performed manually. I'm not sure what the security
>> implications would be from it occurring automatically, especially if
>> during each step there is thorough testing (including from a security
>> team). For our CI/CD pipeline our goal is to have it so every commit
>> can automagically end up on production without human intervention.
>>
>> Currently we use a combination of an initial realm file to be included
>> on startup and also use jq to modify the keycloak-server.json for new
>> keycloak clusters. We don't need to generate realm or client keys as
>> it is included in the initial realm file. That doesn't work for
>> existing systems backed by a database that cannot be thrown away. That
>> kind of leave me with the original option (and hardest) of "write a
>> proprietary liquibase like system built ontop of keycloaks rest api".
>> This is a hard problem to solve
>>
>
> Why proprietary? If we can agree on a design we'll happily accept a
> contribution and maintain it as well.
>
>
>>
>> On Mon, May 23, 2016 at 1:49 PM, Anthony Fryer <anthony.fryer at gmail.com>
>> wrote:
>> > Thanks, I'll check it out.
>> >
>> >
>> > On 05:38, Tue, 24/05/2016 Scott Rossillo <srossillo at smartling.com>
>> wrote:
>> >>
>> >> We use Jose4J[0] to create the keys and then jq[1] to modify the realm
>> >> file.
>> >>
>> >>  See the first line of code here for a super simple example of how to
>> >> generate realm keys:
>> >> https://bitbucket.org/b_c/jose4j/wiki/JWT%20Examples
>> >>
>> >> PS - this may be doable with Keycloak but Jose4J is very lightweight
>> for
>> >> writing a simple script on a CI server.
>> >>
>> >> [0]: https://bitbucket.org/b_c/jose4j
>> >> [1]: https://stedolan.github.io/jq/
>> >>
>> >>
>> >> Scott Rossillo
>> >> Smartling | Senior Software Engineer
>> >> srossillo at smartling.com
>> >>
>> >> On May 21, 2016, at 10:20 PM, Anthony Fryer <anthony.fryer at gmail.com>
>> >> wrote:
>> >>
>> >> Hi Scott,
>> >>
>> >> How do you generate the realm keys when creating the new keycloak dev
>> >> instances?  Do you use a keycloak api or some other way?  I'm
>> interested in
>> >> having a standard realm template that is used to create new realms but
>> would
>> >> need to change the realm keys when importing this template into
>> keycloak.
>> >>
>> >> Cheers,
>> >>
>> >> Anthony
>> >>
>> >> On Sat, May 21, 2016 at 3:43 AM, Scott Rossillo <
>> srossillo at smartling.com>
>> >> wrote:
>> >>>
>> >>> We’re using Keycloak on production, stage/QA, development environments
>> >>> and every developer’s workstation / laptop.
>> >>>
>> >>> While there will always be differing options on how to successfully do
>> >>> change management, we’ve found a very effective method for handling
>> Keycloak
>> >>> provisioning in all environments so that developers don’t need to mess
>> >>> around with. We’re a continuous integration / deployment shop using
>> micro
>> >>> services and everything has to “just work” … I’ll give an overview of
>> our
>> >>> process here but please keep in mind a few things:
>> >>>
>> >>> 1. This approach works for us, I’m not saying it’s the best way
>> >>> 2. We do _not_ allow production config changes to be automated due to
>> >>> security implications
>> >>> 3. We're very opinionated in our approach to configuration management
>> and
>> >>> we don’t ever modify 3rd party software databases directly. We always
>> use
>> >>> APIs.
>> >>>
>> >>> We deploy Keycloak to all environments using Docker images. On
>> developer
>> >>> workstations we use Docker Compose to orchestrate bringing up all
>> services a
>> >>> developer may need, including Keycloak.
>> >>>
>> >>> We have 4 docker images for Keycloak:
>> >>> - Keycloak Base
>> >>>    \- Keycloak HA
>> >>>    \- Keycloak Dev
>> >>> - Keycloak config manager*
>> >>>
>> >>> The base image includes all customizations necessary to bring up a
>> >>> Keycloak instance configured with our modules and themes installed.
>> >>> The HA instance builds off base and configures Keycloak to run as a
>> >>> cluster node. This is used on stage and prod.
>> >>> The dev instance builds off base and includes our realm file. On
>> startup,
>> >>> this instance loads our realm configuration if it’s not already
>> loaded.
>> >>>
>> >>> All docker images are built and published by the CI server and
>> Keycloak
>> >>> HA can be deployed to stage and prod after a clean CI build.
>> >>>
>> >>> Developers are free to add clients for testing, do whatever they want,
>> >>> etc. to their running dev instance. If they want to get back to our
>> stock
>> >>> build, they pull the latest Docker image from our private Docker repo
>> and
>> >>> restart it.
>> >>>
>> >>> Adding clients to stage and prod requires approval and is done by a
>> hand.
>> >>> This is for security reasons. Once a configuration change is detected
>> on
>> >>> stage - say a client is added - our CI server exports the realm from
>> stage,
>> >>> changes the realm keys, and creates a new Keycloak Dev instance with
>> the
>> >>> updated realm file.
>> >>>
>> >>> *A word about configuration management:
>> >>>
>> >>> Obviously, the realm file we generate knows the URLs of staging
>> services,
>> >>> not local or development environment URLs. To overcome this we
>> introduced
>> >>> another Docker based service called the Keycloak configuration
>> manger. It
>> >>> runs on development environments and workstations. It’s responsible
>> for
>> >>> discovering running services and updating Keycloak via its admin
>> endpoints
>> >>> to reflect the proper configuration for the given environment.
>> >>>
>> >>> That’s it. The whole process is automated with the exception of
>> >>> configuration changes to stage and prod which require a security
>> review.
>> >>>
>> >>> Hope this helps. Let me know if you’d like me to elaborate on
>> anything.
>> >>>
>> >>> Best,
>> >>> Scott
>> >>>
>> >>> Scott Rossillo
>> >>> Smartling | Senior Software Engineer
>> >>> srossillo at smartling.com
>> >>>
>> >>> On May 20, 2016, at 1:46 AM, Stian Thorgersen <sthorger at redhat.com>
>> >>> wrote:
>> >>>
>> >>> Firstly, just wanted to highlight that core Keycloak team are devs,
>> not
>> >>> sysadmins/ops guys, so we have limited experience in continuous
>> delivery and
>> >>> maintenance of real production systems. Hence, we'd love input from
>> the
>> >>> community on this.
>> >>>
>> >>> As it stands we don't really have a proper solution. I believe the
>> best
>> >>> you can do at the moment is either using import feature, partial
>> import or
>> >>> admin rest endpoints. Import is not going to work IMO as it requires
>> >>> re-creating the whole realm. Partial import may work, but would work
>> best
>> >>> for new resources rather than modifying existing resources as it does
>> a
>> >>> delete/create operation rather than attempt to modify. With the admin
>> rest
>> >>> endpoints you'd get the best control of what's going on, but
>> obviously that
>> >>> leaves a fair amount of the work.
>> >>>
>> >>> In the future we have an idea of introducing an "import directory" it
>> >>> would be possible to drop json files in here that would add, modify or
>> >>> delete resources (realms, clients, roles, users, whatever). This
>> would allow
>> >>> dropping json files before the server starts and the server would then
>> >>> import on startup. It would also be possible to do this at runtime
>> and new
>> >>> files would be detected at runtime. Finally, we also had an idea of an
>> >>> offline mode to run import of this (it would basically start the
>> server
>> >>> without http listener, import files, then stop, so it could be used
>> in a
>> >>> script/tool). Import is probably not the best name for it, as it would
>> >>> support modify and delete as well as "importing" new things.
>> >>>
>> >>> On 19 May 2016 at 19:53, Jesse Chahal <jessec at stytch.com> wrote:
>> >>>>
>> >>>> Following some of the best practices for continuous Integration and
>> >>>> continuous delivery there needs to be environments for build, test,
>> >>>> and production. This would mean that following these practices would
>> >>>> require you to have multiple versions of keycloak at different stages
>> >>>> of development cycle. Some of these environments might not have
>> >>>> important persistent data while others might. In order to have builds
>> >>>> transition from one environment to another there may be configuration
>> >>>> changes required for a build to be valid. This is especially true
>> when
>> >>>> new services (openid clients) are being added or "default" accounts.
>> >>>> I'm trying to come up with a scripted way of updating keycloak
>> >>>> instances that are backed up by an RDMS. This may include adding new
>> >>>> clients, adding new users, updating realm config, etc... Originally I
>> >>>> was planning on simply exporting the realm config and importing it
>> >>>> every time keycloak starts. If I enabled the OVERWRITE option I might
>> >>>> overwrite things that I do not want overridden. This is especially
>> >>>> true if there is some config that differ's based on whether it is a
>> >>>> build, test, or production instance. If I don't enable it then it is
>> >>>> only useful for new/blank keycloak environments. I considered using
>> >>>> liquibase but since I do not have control of schema changes created
>> by
>> >>>> the keycloak team I might run into issues with my liquibase file not
>> >>>> being valid after a migration/liquibase update by the keycloak team
>> as
>> >>>> my liquibase file would run after keycloak's does. There might also
>> be
>> >>>> some other unknown issues our liquibase changes conflicting somehow
>> >>>> with keycloak's liquibase changes. I've also considered writing my
>> own
>> >>>> updater tool using a scripting language (python/ruby) that calls
>> >>>> keycloak's rest api. The issues with this mechanism is it feels like
>> I
>> >>>> am recreating the wheel as well as not being able to find good
>> >>>> documentation on keycloak's openid endpoints/url's used for different
>> >>>> oauth2 flows. Even if I did find this documentation it would also
>> >>>> require me to find a good openid client for the scripting language.
>> >>>> This doesn't matter for our normal clients as they simply use the
>> >>>> keycloak subsystems and adapters instead. I've also looked at
>> commonly
>> >>>> used server configuration software such as chef, puppet, and ansible.
>> >>>> I don't see a good solution using any of those tools yet either. What
>> >>>> have other people done for cases like this? Please don't tell me
>> there
>> >>>> is someone who is doing this all manually because that doesn't work
>> in
>> >>>> modern software development.
>> >>>>
>> >>>> - doesn't accidentally delete users
>> >>>> - doesn't accidentally delete clients
>> >>>> - doesn't invalidate sessions (optional)
>> >>>> - works to bring up new, correctly configured, keycloak instances
>> >>>> - handles applying updates to existing keycloak instances
>> >>>> - can handle minor differences between keycloak instances (build,
>> >>>> test, production) when updating
>> >>>> - preferably can work well in rolling deployment scenario's.
>> >>>> -- I hope the keycloak team is taking these into consideration when
>> >>>> doing database migration between 1-2 releases. It would be nice if
>> >>>> they set some specific rules for rolling updates between versions
>> (aka
>> >>>> backwards breaking changes)
>> >>>> _______________________________________________
>> >>>> keycloak-user mailing list
>> >>>> keycloak-user at lists.jboss.org
>> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> keycloak-user mailing list
>> >>> keycloak-user at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> keycloak-user mailing list
>> >>> keycloak-user at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>> >>
>> >>
>> >>
>> >
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160526/6ab4cca4/attachment-0001.html 