[keycloak-user] How to apply updates to keycloak instances

[Stian Thorgersen]

Adding list back..

I don't see much value in a solution that doesn't also consider changes
done directly through admin console and/or admin endpoints. A proper
solution would use something along the lines of Liquibase/Git to have all
changes versioned and applied serially. That way they can be reproduced
fully.

On 10 June 2016 at 21:03, Jesse Chahal <jessec at stytch.com> wrote:

> I've been thinking about this problem for awhile and so far the
> solutions that I come up with all require that keycloak keeps tracks
> of changes in a database table (exactly how it works for liquibase).
> The GUI has a partial import feature. I haven't used it too
> extensively but I believe it probably does some sort of JSONtoPOJO
> serialization in order to figure out what the partial update it needs
> to be doing. Maybe we could add unique id identifiers to the
> existing/exported JSON files and have keycloaks import features
> determine whether the JSON file had already been applied or not. If
> there is a rest api for this as well then building an external cli or
> GUI tool would be much more feasible. Scott's solution requires either
> the external app to know the state of keycloak or keycloak's state to
> be blank. Its the best that could have been done with keycloak as it
> is now. Anyone have any comments regarding this possible solution?
>
> On Thu, May 26, 2016 at 11:19 PM, Stian Thorgersen <sthorger at redhat.com>
> wrote:
> > Can you give me some examples of issues around Dockerized deployments and
> > services that are located at runtime (do you mean services that are
> > provisioned at runtime?)?
> >
> > On 26 May 2016 at 19:47, Scott Rossillo <srossillo at smartling.com> wrote:
> >>
> >> Stian, that’s fair, it does solve the OP's CI/CD problem when moving in
> >> the dev -> stage -> prod direction.
> >>
> >> Scott Rossillo
> >> Smartling | Senior Software Engineer
> >> srossillo at smartling.com
> >>
> >> On May 26, 2016, at 1:41 PM, Stian Thorgersen <sthorger at redhat.com>
> wrote:
> >>
> >>
> >>
> >> On 26 May 2016 at 19:11, Scott Rossillo <srossillo at smartling.com>
> wrote:
> >>>
> >>> I guess it’s a matter of requirements, but with micro service
> >>> architectures there’s usually some sort of discovery mechanism
> required to
> >>> locale services at runtime. Netflix offers Eureka and then there’s
> etcd from
> >>> CoreOS that’s being used by Kubernetes. My point is that even if
> Keycloak
> >>> devs build some sort of way of picking up changes from the filesystem
> on
> >>> startup, that doesn’t solve all use cases.
> >>
> >>
> >> The problem is continuous integration right, and pushing changes from a
> >> test environment into production? So you need a reliable way to apply
> >> changes to both environments.
> >>
> >>>
> >>>
> >>> It doesn’t solve issues with Dockerized deployments and it doesn’t
> solve
> >>> the issue where services have to be located at runtime
> >>
> >>
> >> What are the issues it doesn't solve?
> >>
> >>>
> >>>
> >>> Scott Rossillo
> >>> Smartling | Senior Software Engineer
> >>> srossillo at smartling.com
> >>>
> >>> On May 26, 2016, at 2:27 AM, Stian Thorgersen <sthorger at redhat.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> On 26 May 2016 at 02:15, Jesse Chahal <jessec at stytch.com> wrote:
> >>>>
> >>>> @Stian
> >>>> The approach described sounds similar to liquibase to me but with json
> >>>> and specific to keycloak. I feel like a lot of possible bugs could
> >>>> arise from this approach or at least quite a few feature requests.
> >>>> Would each json file only contain a single change? Would order matter
> >>>> in how they get applied if you put a bunch of json files in this
> >>>> directory at once? Can the same file be applied multiple times? These
> >>>> are the kind of issues I would expect to come up with this type of
> >>>> change management system. When I mentioned write our own tool/script
> >>>> to do it I was kind of thinking of a writing a liquibase like system
> >>>> that calls keycloak's rest api.
> >>>
> >>>
> >>> We haven't figured out all the details, but what you are proposing
> sounds
> >>> better. A single document that lists all changes, that can also import
> other
> >>> files, sorts out the ordering and we could add same type of ids as
> Liquibase
> >>> does to changesets.
> >>>
> >>> You could write it to use the rest api, then use a separate db to store
> >>> what changes have been applied, but would be better if Keycloak deals
> with
> >>> loading the changes directly as it can write to the db what changes
> have
> >>> been applied.
> >>>
> >>> One big issue is what happens if manual changes are done through the
> >>> admin console. One though (although probably very tricky to get right)
> is
> >>> that changes done through the admin console is added to the changeset.
> >>>
> >>>>
> >>>>
> >>>> @ Scott
> >>>> If I would compare the solution you mentioned to one of the options I
> >>>> listed in my original question  "I've also considered writing my own
> >>>> updater tool using a scripting language (python/ruby) that calls
> >>>> keycloak's rest api." The worrying thing to me is that there is
> >>>> another piece of code that needs to maintained by our company and
> >>>> requires quite a bit of knowledge of keycloak's rest api. There would
> >>>> probably need to be some serious thought put into the architecture of
> >>>> the tool as well. Without a doubt it does provide the most control. We
> >>>> also live by a different methodology in regards to updating production
> >>>> clusters. From our perspective it is more of an issue to update
> >>>> manually as it becomes much easier to miss a step or in someway screw
> >>>> up if steps are performed manually. I'm not sure what the security
> >>>> implications would be from it occurring automatically, especially if
> >>>> during each step there is thorough testing (including from a security
> >>>> team). For our CI/CD pipeline our goal is to have it so every commit
> >>>> can automagically end up on production without human intervention.
> >>>>
> >>>> Currently we use a combination of an initial realm file to be included
> >>>> on startup and also use jq to modify the keycloak-server.json for new
> >>>> keycloak clusters. We don't need to generate realm or client keys as
> >>>> it is included in the initial realm file. That doesn't work for
> >>>> existing systems backed by a database that cannot be thrown away. That
> >>>> kind of leave me with the original option (and hardest) of "write a
> >>>> proprietary liquibase like system built ontop of keycloaks rest api".
> >>>> This is a hard problem to solve
> >>>
> >>>
> >>> Why proprietary? If we can agree on a design we'll happily accept a
> >>> contribution and maintain it as well.
> >>>
> >>>>
> >>>>
> >>>> On Mon, May 23, 2016 at 1:49 PM, Anthony Fryer <
> anthony.fryer at gmail.com>
> >>>> wrote:
> >>>> > Thanks, I'll check it out.
> >>>> >
> >>>> >
> >>>> > On 05:38, Tue, 24/05/2016 Scott Rossillo <srossillo at smartling.com>
> >>>> > wrote:
> >>>> >>
> >>>> >> We use Jose4J[0] to create the keys and then jq[1] to modify the
> >>>> >> realm
> >>>> >> file.
> >>>> >>
> >>>> >>  See the first line of code here for a super simple example of how
> to
> >>>> >> generate realm keys:
> >>>> >> https://bitbucket.org/b_c/jose4j/wiki/JWT%20Examples
> >>>> >>
> >>>> >> PS - this may be doable with Keycloak but Jose4J is very
> lightweight
> >>>> >> for
> >>>> >> writing a simple script on a CI server.
> >>>> >>
> >>>> >> [0]: https://bitbucket.org/b_c/jose4j
> >>>> >> [1]: https://stedolan.github.io/jq/
> >>>> >>
> >>>> >>
> >>>> >> Scott Rossillo
> >>>> >> Smartling | Senior Software Engineer
> >>>> >> srossillo at smartling.com
> >>>> >>
> >>>> >> On May 21, 2016, at 10:20 PM, Anthony Fryer <
> anthony.fryer at gmail.com>
> >>>> >> wrote:
> >>>> >>
> >>>> >> Hi Scott,
> >>>> >>
> >>>> >> How do you generate the realm keys when creating the new keycloak
> dev
> >>>> >> instances?  Do you use a keycloak api or some other way?  I'm
> >>>> >> interested in
> >>>> >> having a standard realm template that is used to create new realms
> >>>> >> but would
> >>>> >> need to change the realm keys when importing this template into
> >>>> >> keycloak.
> >>>> >>
> >>>> >> Cheers,
> >>>> >>
> >>>> >> Anthony
> >>>> >>
> >>>> >> On Sat, May 21, 2016 at 3:43 AM, Scott Rossillo
> >>>> >> <srossillo at smartling.com>
> >>>> >> wrote:
> >>>> >>>
> >>>> >>> We’re using Keycloak on production, stage/QA, development
> >>>> >>> environments
> >>>> >>> and every developer’s workstation / laptop.
> >>>> >>>
> >>>> >>> While there will always be differing options on how to
> successfully
> >>>> >>> do
> >>>> >>> change management, we’ve found a very effective method for
> handling
> >>>> >>> Keycloak
> >>>> >>> provisioning in all environments so that developers don’t need to
> >>>> >>> mess
> >>>> >>> around with. We’re a continuous integration / deployment shop
> using
> >>>> >>> micro
> >>>> >>> services and everything has to “just work” … I’ll give an overview
> >>>> >>> of our
> >>>> >>> process here but please keep in mind a few things:
> >>>> >>>
> >>>> >>> 1. This approach works for us, I’m not saying it’s the best way
> >>>> >>> 2. We do _not_ allow production config changes to be automated due
> >>>> >>> to
> >>>> >>> security implications
> >>>> >>> 3. We're very opinionated in our approach to configuration
> >>>> >>> management and
> >>>> >>> we don’t ever modify 3rd party software databases directly. We
> >>>> >>> always use
> >>>> >>> APIs.
> >>>> >>>
> >>>> >>> We deploy Keycloak to all environments using Docker images. On
> >>>> >>> developer
> >>>> >>> workstations we use Docker Compose to orchestrate bringing up all
> >>>> >>> services a
> >>>> >>> developer may need, including Keycloak.
> >>>> >>>
> >>>> >>> We have 4 docker images for Keycloak:
> >>>> >>> - Keycloak Base
> >>>> >>>    \- Keycloak HA
> >>>> >>>    \- Keycloak Dev
> >>>> >>> - Keycloak config manager*
> >>>> >>>
> >>>> >>> The base image includes all customizations necessary to bring up a
> >>>> >>> Keycloak instance configured with our modules and themes
> installed.
> >>>> >>> The HA instance builds off base and configures Keycloak to run as
> a
> >>>> >>> cluster node. This is used on stage and prod.
> >>>> >>> The dev instance builds off base and includes our realm file. On
> >>>> >>> startup,
> >>>> >>> this instance loads our realm configuration if it’s not already
> >>>> >>> loaded.
> >>>> >>>
> >>>> >>> All docker images are built and published by the CI server and
> >>>> >>> Keycloak
> >>>> >>> HA can be deployed to stage and prod after a clean CI build.
> >>>> >>>
> >>>> >>> Developers are free to add clients for testing, do whatever they
> >>>> >>> want,
> >>>> >>> etc. to their running dev instance. If they want to get back to
> our
> >>>> >>> stock
> >>>> >>> build, they pull the latest Docker image from our private Docker
> >>>> >>> repo and
> >>>> >>> restart it.
> >>>> >>>
> >>>> >>> Adding clients to stage and prod requires approval and is done by
> a
> >>>> >>> hand.
> >>>> >>> This is for security reasons. Once a configuration change is
> >>>> >>> detected on
> >>>> >>> stage - say a client is added - our CI server exports the realm
> from
> >>>> >>> stage,
> >>>> >>> changes the realm keys, and creates a new Keycloak Dev instance
> with
> >>>> >>> the
> >>>> >>> updated realm file.
> >>>> >>>
> >>>> >>> *A word about configuration management:
> >>>> >>>
> >>>> >>> Obviously, the realm file we generate knows the URLs of staging
> >>>> >>> services,
> >>>> >>> not local or development environment URLs. To overcome this we
> >>>> >>> introduced
> >>>> >>> another Docker based service called the Keycloak configuration
> >>>> >>> manger. It
> >>>> >>> runs on development environments and workstations. It’s
> responsible
> >>>> >>> for
> >>>> >>> discovering running services and updating Keycloak via its admin
> >>>> >>> endpoints
> >>>> >>> to reflect the proper configuration for the given environment.
> >>>> >>>
> >>>> >>> That’s it. The whole process is automated with the exception of
> >>>> >>> configuration changes to stage and prod which require a security
> >>>> >>> review.
> >>>> >>>
> >>>> >>> Hope this helps. Let me know if you’d like me to elaborate on
> >>>> >>> anything.
> >>>> >>>
> >>>> >>> Best,
> >>>> >>> Scott
> >>>> >>>
> >>>> >>> Scott Rossillo
> >>>> >>> Smartling | Senior Software Engineer
> >>>> >>> srossillo at smartling.com
> >>>> >>>
> >>>> >>> On May 20, 2016, at 1:46 AM, Stian Thorgersen <
> sthorger at redhat.com>
> >>>> >>> wrote:
> >>>> >>>
> >>>> >>> Firstly, just wanted to highlight that core Keycloak team are
> devs,
> >>>> >>> not
> >>>> >>> sysadmins/ops guys, so we have limited experience in continuous
> >>>> >>> delivery and
> >>>> >>> maintenance of real production systems. Hence, we'd love input
> from
> >>>> >>> the
> >>>> >>> community on this.
> >>>> >>>
> >>>> >>> As it stands we don't really have a proper solution. I believe the
> >>>> >>> best
> >>>> >>> you can do at the moment is either using import feature, partial
> >>>> >>> import or
> >>>> >>> admin rest endpoints. Import is not going to work IMO as it
> requires
> >>>> >>> re-creating the whole realm. Partial import may work, but would
> work
> >>>> >>> best
> >>>> >>> for new resources rather than modifying existing resources as it
> >>>> >>> does a
> >>>> >>> delete/create operation rather than attempt to modify. With the
> >>>> >>> admin rest
> >>>> >>> endpoints you'd get the best control of what's going on, but
> >>>> >>> obviously that
> >>>> >>> leaves a fair amount of the work.
> >>>> >>>
> >>>> >>> In the future we have an idea of introducing an "import directory"
> >>>> >>> it
> >>>> >>> would be possible to drop json files in here that would add,
> modify
> >>>> >>> or
> >>>> >>> delete resources (realms, clients, roles, users, whatever). This
> >>>> >>> would allow
> >>>> >>> dropping json files before the server starts and the server would
> >>>> >>> then
> >>>> >>> import on startup. It would also be possible to do this at runtime
> >>>> >>> and new
> >>>> >>> files would be detected at runtime. Finally, we also had an idea
> of
> >>>> >>> an
> >>>> >>> offline mode to run import of this (it would basically start the
> >>>> >>> server
> >>>> >>> without http listener, import files, then stop, so it could be
> used
> >>>> >>> in a
> >>>> >>> script/tool). Import is probably not the best name for it, as it
> >>>> >>> would
> >>>> >>> support modify and delete as well as "importing" new things.
> >>>> >>>
> >>>> >>> On 19 May 2016 at 19:53, Jesse Chahal <jessec at stytch.com> wrote:
> >>>> >>>>
> >>>> >>>> Following some of the best practices for continuous Integration
> and
> >>>> >>>> continuous delivery there needs to be environments for build,
> test,
> >>>> >>>> and production. This would mean that following these practices
> >>>> >>>> would
> >>>> >>>> require you to have multiple versions of keycloak at different
> >>>> >>>> stages
> >>>> >>>> of development cycle. Some of these environments might not have
> >>>> >>>> important persistent data while others might. In order to have
> >>>> >>>> builds
> >>>> >>>> transition from one environment to another there may be
> >>>> >>>> configuration
> >>>> >>>> changes required for a build to be valid. This is especially true
> >>>> >>>> when
> >>>> >>>> new services (openid clients) are being added or "default"
> >>>> >>>> accounts.
> >>>> >>>> I'm trying to come up with a scripted way of updating keycloak
> >>>> >>>> instances that are backed up by an RDMS. This may include adding
> >>>> >>>> new
> >>>> >>>> clients, adding new users, updating realm config, etc...
> Originally
> >>>> >>>> I
> >>>> >>>> was planning on simply exporting the realm config and importing
> it
> >>>> >>>> every time keycloak starts. If I enabled the OVERWRITE option I
> >>>> >>>> might
> >>>> >>>> overwrite things that I do not want overridden. This is
> especially
> >>>> >>>> true if there is some config that differ's based on whether it
> is a
> >>>> >>>> build, test, or production instance. If I don't enable it then it
> >>>> >>>> is
> >>>> >>>> only useful for new/blank keycloak environments. I considered
> using
> >>>> >>>> liquibase but since I do not have control of schema changes
> created
> >>>> >>>> by
> >>>> >>>> the keycloak team I might run into issues with my liquibase file
> >>>> >>>> not
> >>>> >>>> being valid after a migration/liquibase update by the keycloak
> team
> >>>> >>>> as
> >>>> >>>> my liquibase file would run after keycloak's does. There might
> also
> >>>> >>>> be
> >>>> >>>> some other unknown issues our liquibase changes conflicting
> somehow
> >>>> >>>> with keycloak's liquibase changes. I've also considered writing
> my
> >>>> >>>> own
> >>>> >>>> updater tool using a scripting language (python/ruby) that calls
> >>>> >>>> keycloak's rest api. The issues with this mechanism is it feels
> >>>> >>>> like I
> >>>> >>>> am recreating the wheel as well as not being able to find good
> >>>> >>>> documentation on keycloak's openid endpoints/url's used for
> >>>> >>>> different
> >>>> >>>> oauth2 flows. Even if I did find this documentation it would also
> >>>> >>>> require me to find a good openid client for the scripting
> language.
> >>>> >>>> This doesn't matter for our normal clients as they simply use the
> >>>> >>>> keycloak subsystems and adapters instead. I've also looked at
> >>>> >>>> commonly
> >>>> >>>> used server configuration software such as chef, puppet, and
> >>>> >>>> ansible.
> >>>> >>>> I don't see a good solution using any of those tools yet either.
> >>>> >>>> What
> >>>> >>>> have other people done for cases like this? Please don't tell me
> >>>> >>>> there
> >>>> >>>> is someone who is doing this all manually because that doesn't
> work
> >>>> >>>> in
> >>>> >>>> modern software development.
> >>>> >>>>
> >>>> >>>> - doesn't accidentally delete users
> >>>> >>>> - doesn't accidentally delete clients
> >>>> >>>> - doesn't invalidate sessions (optional)
> >>>> >>>> - works to bring up new, correctly configured, keycloak instances
> >>>> >>>> - handles applying updates to existing keycloak instances
> >>>> >>>> - can handle minor differences between keycloak instances (build,
> >>>> >>>> test, production) when updating
> >>>> >>>> - preferably can work well in rolling deployment scenario's.
> >>>> >>>> -- I hope the keycloak team is taking these into consideration
> when
> >>>> >>>> doing database migration between 1-2 releases. It would be nice
> if
> >>>> >>>> they set some specific rules for rolling updates between versions
> >>>> >>>> (aka
> >>>> >>>> backwards breaking changes)
> >>>> >>>> _______________________________________________
> >>>> >>>> keycloak-user mailing list
> >>>> >>>> keycloak-user at lists.jboss.org
> >>>> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-user
> >>>> >>>
> >>>> >>>
> >>>> >>> _______________________________________________
> >>>> >>> keycloak-user mailing list
> >>>> >>> keycloak-user at lists.jboss.org
> >>>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-user
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>> _______________________________________________
> >>>> >>> keycloak-user mailing list
> >>>> >>> keycloak-user at lists.jboss.org
> >>>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-user
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >
> >>>
> >>>
> >>>
> >>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160613/bdc883d5/attachment-0001.html