[keycloak-dev] Client Registration CLI tool

[Stian Thorgersen]

On 30 June 2016 at 08:08, Bruno Oliveira <bruno at abstractj.org> wrote:

> What I suggested was based on what I saw at our current proposal:
>
> $ kcreg login -r master -u manage-client
> Server to connect to (http://localhost:8080/auth):
> Password:
>
> If you want to automate it for example in a shell script, unless I'm
> mistaken, you have to provide user's password or obtain initial access
> token through console. If we're assuming that's the default
> flow for automated scripts should be obtaining an initial access token
> from console first, that's fine.
>

Yes, the password is just for manual use and wouldn't be cached or anything
like that.


>
> Flow for encryption:
>
> 1. Admin client retrieve the public key from the server
> 2. Admin client encrypt's whatever data is sensitive
> 3. Server decrypt's it
>

Isn't that only going to hide the password? The encrypted password could
still be used as the server would accept/decrypt it.


>
> On 2016-06-30, Stian Thorgersen wrote:
> > I can't see why we should consider it though:
> >
> > a) We have service account support, initial access tokens, offline
> tokens,
> > etc.. All designed so credentials don't need to be stored. Using a user
> > account directly is horrible as not only are you potentially exposing the
> > credentials, you're also giving all permissions of the user to the
> > automation. Using tokens or service account you can limit what
> permissions
> > the automation has.
> > b) If the credentials are encrypted, you'll also need to unlock the
> > vault/encrypted file. How would you suggest we do that? I can't see the
> > full flow here.
> >
> > On 30 June 2016 at 07:15, Bruno Oliveira <bruno at abstractj.org> wrote:
> >
> > > Yes, the encrypted data goes into public repository with .travis.yaml
> file.
> > > Only the Travis server is able to decrypt such data and validate
> it[1][2].
> > >
> > > Like I mentioned, not a blocker, but maybe something to take into
> > > consideration.
> > >
> > > [1] -
> > > https://docs.travis-ci.com/user/encryption-keys/#Notifications-Example
> > > [2] -
> https://github.com/twbs/bootstrap/blob/master/.travis.yml#L30-L32
> > >
> > > On 2016-06-30, Stian Thorgersen wrote:
> > > > The encryption support in Travis isn't that so you can encrypt
> sensitive
> > > > details that goes into .travis.yaml which is public through the repo?
> > > >
> > > > On 30 June 2016 at 06:52, Stian Thorgersen <sthorger at redhat.com>
> wrote:
> > > >
> > > > > I don't see the need for that. For the client registration CLI we
> will
> > > > > support initial access tokens as well as service accounts with
> > > pub/priv key
> > > > > authentication. For admin cli we'll support service accounts. No
> one
> > > should
> > > > > be using username/password combined with automated jobs.
> > > > >
> > > > > On 29 June 2016 at 21:49, Bruno Oliveira <bruno at abstractj.org>
> wrote:
> > > > >
> > > > >> I'm not sure if it's part of the scope for now. But
> > > > >> thinking about situations where you have to automate jobs plus
> provide
> > > > >> credentials without exposing them.
> > > > >>
> > > > >> My suggestion, even if it's not part of the scope for now is to
> > > encrypt
> > > > >> it,
> > > > >> like travis does[1]. I know that our plan is to deal of access
> token,
> > > > >> but would be nice to not expose credentials not even a single
> time.
> > > > >>
> > > > >>
> > > > >> [1] - https://docs.travis-ci.com/user/encryption-keys/
> > > > >>
> > > > >> On 2016-06-29, Stian Thorgersen wrote:
> > > > >> > On 28 June 2016 at 11:40, Marko Strukelj <mstrukel at redhat.com>
> > > wrote:
> > > > >> >
> > > > >> > >
> > > > >> > >
> > > > >> > > On Tue, Jun 28, 2016 at 7:35 AM, Stian Thorgersen <
> > > > >> sthorger at redhat.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > >>
> > > > >> > >>
> > > > >> > >> On 27 June 2016 at 21:26, John Dennis <jdennis at redhat.com>
> > > wrote:
> > > > >> > >>
> > > > >> > >>> On 06/27/2016 07:48 AM, Marko Strukelj wrote:
> > > > >> > >>> > I've started work on Client Registration CLI tool. As a
> first
> > > > >> step,
> > > > >> > >>> here
> > > > >> > >>> > is a design document describing how I imagine the tool
> would
> > > be
> > > > >> used.
> > > > >> > >>> >
> > > > >> > >>> >
> > > > >> > >>> >
> > > > >> > >>>
> > > > >>
> > >
> https://docs.google.com/document/d/18SoZ34sY_k7N8ae-WDsvo7QeI-cHkpTURIlUk5dpIhU/edit?usp=sharing
> > > > >> > >>> >
> > > > >> > >>> >
> > > > >> > >>> > I'll use this document as a spec / guide as I implement
> the
> > > client
> > > > >> > >>> tool.
> > > > >> > >>> >
> > > > >> > >>> > Within days I'll also send a link to initial ideas for
> Admin
> > > > >> Client
> > > > >> > >>> tool
> > > > >> > >>> > which in principle should allow administrator to configure
> > > > >> everything
> > > > >> > >>> > that can otherwise be done through Admin Console.
> > > > >> > >>> >
> > > > >> > >>> > Any feedback welcome.
> > > > >> > >>>
> > > > >> > >>> FWIW we've already written a client registration tool for
> > > Keycloak.
> > > > >> At
> > > > >> > >>> the moment it is specifically targeted for SAML clients (SP,
> > > Service
> > > > >> > >>> Provider) in Apache HTTPD but we have plans to extend it to
> > > OIDC.
> > > > >> > >>>
> > > > >> > >>> It is currently in Fedora and will also ship in OSP.
> > > > >> > >>>
> > > > >> > >>> It is hosted here:
> > > > >> > >>> https://github.com/jdennis/keycloak-httpd-client-install
> > > > >> > >>>
> > > > >> > >>> The man page for it (formatted for HTML) can be found here:
> > > > >> > >>>
> > > > >>
> > >
> https://jdennis.fedorapeople.org/doc/keycloak-httpd-client-install.html
> > > > >> > >>>
> > > > >> > >>> The man page discusses 3 different ways you can authenticate
> > > and 2
> > > > >> > >>> different ways client registration can be performed.
> > > > >> > >>>
> > > > >> > >>> I have a lot of experience with Keycloak client registration
> > > tools
> > > > >> and
> > > > >> > >>> have worked through many issues, I'm happy to share my
> > > experience.
> > > > >> > >>>
> > > > >> > >>> Here are some thoughts/issues you may want to take into
> account:
> > > > >> > >>>
> > > > >> > >>> * The tool must be capable of running without interactivity
> as
> > > part
> > > > >> of a
> > > > >> > >>> scripted installation task.
> > > > >> > >>>
> > > > >> > >>> * It should not depend on a home directory being available.
> > > > >> > >>>
> > > > >> > >>> * If a home directory is utilized how will you disambiguate
> any
> > > > >> stored
> > > > >> > >>> state belonging to a script that is run by different
> processes
> > > but
> > > > >> under
> > > > >> > >>> the same user (possibly simultaneously)? To clarify, many
> > > install
> > > > >> tools
> > > > >> > >>> run as the root user or some other admin user. Each
> invocation
> > > of
> > > > >> these
> > > > >> > >>> install tools can be run with entirely different parameters
> and
> > > may
> > > > >> > >>> execute either in parallel or partially overlapping in time.
> > > > >> > >>>
> > > > >> > >>
> > > > >> > > Maybe I should have included this link in the design document
> to
> > > make
> > > > >> it
> > > > >> > > clear to everyone what Client Registration this tool is for:
> > > > >> > >
> > > > >>
> > >
> http://keycloak.github.io/docs/userguide/keycloak-server/html/client-registration.html
> > > > >> > >
> > > > >> > > It's a REST API defined by specs, and is separate from Admin
> REST
> > > API.
> > > > >> > >
> > > > >> > > About using home directory, the way I see it - you either a)
> > > specify
> > > > >> all
> > > > >> > > the state when executing a command, or b) you have a mechanism
> > > that
> > > > >> allows
> > > > >> > > the concept of 'session' between command invocations.
> > > > >> > >
> > > > >> > > If you use the first approach (a) then on each invocation of
> the
> > > > >> command
> > > > >> > > you have to specify either username:password, or a token. The
> > > client
> > > > >> > > registration specification defines workflow for Initial Access
> > > > >> Tokens, and
> > > > >> > > Registration Access Tokens, which require to automatically
> > > intercept a
> > > > >> > > newly issued token after each CRUD operation, and save it for
> any
> > > > >> > > subsequent operation on the same client resource. I can't see
> how
> > > this
> > > > >> > > could be achieved by using the first approach.
> > > > >> > >
> > > > >> > > For the second approach (b) you need a way to communicate
> > > 'session'
> > > > >> state.
> > > > >> > > The state we are saving are just tokens associated with
> current
> > > user
> > > > >> or
> > > > >> > > specific clients, or specific grants. Looks to me that if
> multiple
> > > > >> parallel
> > > > >> > > sessions are in collision about these tokens then the cli tool
> > > itself
> > > > >> might
> > > > >> > > be used the wrong way. Namely, once the client authenticates
> with
> > > a
> > > > >> login,
> > > > >> > > access token and refresh token are cached. Multiple client
> > > instances
> > > > >> can
> > > > >> > > use the same access token, and the same refresh token. A
> thing to
> > > > >> maybe be
> > > > >> > > careful about is just properly locking the file when making
> > > changes
> > > > >> to it.
> > > > >> > > For initial access token you have to explicitly add it, and
> > > assign it
> > > > >> an
> > > > >> > > alias - you can use any random value there if you want. For
> > > > >> registration
> > > > >> > > access token they are automatically associated with initial
> token
> > > > >> they were
> > > > >> > > initiated from - again there should be no collision.
> > > > >> > >
> > > > >> >
> > > > >> > I like the option to have two approaches as there are two
> use-cases.
> > > > >> One is
> > > > >> > manually registering for example during development or when
> manually
> > > > >> > configuring an application to use Keycloak. Another is scripted.
> > > Even
> > > > >> for
> > > > >> > scripted you may quite likely want to just add service account
> > > > >> credentials
> > > > >> > or initial access token directly to ~/.keycloak/ rather than
> pass
> > > these
> > > > >> to
> > > > >> > the commands.
> > > > >> >
> > > > >> > Registration access tokens are associated with a client, not an
> > > initial
> > > > >> > access token. Also, remember the registration access token is
> > > changed on
> > > > >> > updates.
> > > > >> >
> > > > >> >
> > > > >> > >
> > > > >> > > What alternative mechanism would you suggest for storing
> 'session'
> > > > >> info?
> > > > >> > > We want to support Windows as well so it can't be Unix / Bash
> > > > >> specific.
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > >>
> > > > >> > >>> * The tool should be idempotent.
> > > > >> > >>>
> > > > >> > >>> * You suggest storing tokens in a cache, how do you plan on
> > > > >> handling the
> > > > >> > >>> case where a token expires before all operations are
> complete?
> > > > >> > >>>
> > > > >> > >>> * We also initially took the approach of caching tokens but
> > > > >> discovered
> > > > >> > >>> the complexity did not justify the minimal cost of
> obtaining a
> > > new
> > > > >> token
> > > > >> > >>> for each invocation. This greatly simplified the code with
> very
> > > > >> little
> > > > >> > >>> performance impact.
> > > > >> > >>>
> > > > >> > >>> * You do not mention what type of client you're
> registering. I'm
> > > > >> > >>> assuming it's OpenID but SAML clients (SP) are equally
> > > important.
> > > > >> The
> > > > >> > >>> tool must be able to handle both.
> > > > >> > >>>
> > > > >> > >>
> > > > >> > >> Marko is probably referring to the Keycloak client
> > > representation,
> > > > >> which
> > > > >> > >> can be either OpenID or SAML. However, we also need to
> support
> > > OpenID
> > > > >> > >> Connect client descriptions as well as SAML entity
> descriptors as
> > > > >> both are
> > > > >> > >> supported by client reg services.
> > > > >> > >>
> > > > >> > >
> > > > >> > > The CLI needs to know which of the client registration
> providers
> > > (REST
> > > > >> > > endpoints) to use - there are four as described in the Client
> > > > >> Registration
> > > > >> > > documentation (
> > > > >> > >
> > > > >>
> > >
> http://keycloak.github.io/docs/userguide/keycloak-server/html/client-registration.html
> > > > >> > > )
> > > > >> > >
> > > > >> > > Ideally the input format of the file could be recognised as
> only
> > > > >> > > appropriate for one of these providers, and the correct
> provider
> > > then
> > > > >> > > automatically used. But maybe we need a way to explicitly
> tell the
> > > > >> tool
> > > > >> > > what provider to use. For example:
> > > > >> > >
> > > > >> > > kc new --type default --name test-app --enabled true
> --base-url
> > > > >> > > http://localhost:8480/test-app --redirect-uri '
> > > > >> > > http://localhost:8480/test-app/*' --admin-url
> > > > >> > > http://localhost:8480/test-app/logout --secret password | kc
> > > create
> > > > >> > > --type default -f -
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > Having to set --type for both creating a description (kc
> new), and
> > > > >> > > pushing it to the server (kc create) is not ideal.
> > > > >> > >
> > > > >> >
> > > > >> > We can detect the difference between Keycloak client rep, SAML
> and
> > > OIDC
> > > > >> > json that's pretty trivial and we should do that. However, there
> > > needs
> > > > >> to
> > > > >> > be a way to specify the provider as well as there could be
> custom
> > > > >> providers
> > > > >> > added.
> > > > >> >
> > > > >> > For getting the client it should by default return Keycloak
> client
> > > > >> > representation, but we need an option to be able to specify the
> > > > >> provider so
> > > > >> > it can return the client installation file instead.
> > > > >> >
> > > > >> > With regards to creating by passing arguments rather than a file
> > > that
> > > > >> > should only support client representation files.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > >
> > > > >> > >
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>>
> > > > >> > >>> * I don't see anything in your document on how to specify
> the
> > > SAML
> > > > >> > >>> metadata.
> > > > >> > >>>
> > > > >> > >>
> > > > >> > > Instead of piping in my-client.json, you would pipe in
> > > > >> my-client-saml.xml,
> > > > >> > > possibly requiring an extra --type specifier as described
> above.
> > > > >> > >
> > > > >> > >
> > > > >> > >>
> > > > >> > >>> * I don't see anything in your document on how the user
> > > modifies the
> > > > >> > >>> client. It appears as if you are retrieving a
> > > ClientRepresentation
> > > > >> JSON
> > > > >> > >>> document and expecting the user to edit it in a text editor
> > > which
> > > > >> will
> > > > >> > >>> then be sent back. That won't work for non-interactive
> > > installs. It
> > > > >> also
> > > > >> > >>> presumes the user knows how to read and modify the JSON.
> > > > >> > >>>
> > > > >> > >>
> > > > >> > >> It would be nice to be able to set specific fields without
> > > having to
> > > > >> > >> modify JSON. We discussed that for the Admin CLI, but we
> should
> > > > >> probably
> > > > >> > >> also add it to the client reg CLI
> > > > >> > >>
> > > > >> > >
> > > > >> > > It would be very valuable to see some of the usecases, what
> kind
> > > of
> > > > >> > > changes people do on existing clients.
> > > > >> > >
> > > > >> >
> > > > >> > We should support anything that is in client representation. It
> > > should
> > > > >> just
> > > > >> > be a generic way to change json fields. For example:
> > > > >> >
> > > > >> > # kcreg update <client id> --set enabled=false --set baseUrl=
> > > > >> > http://new-url/myapp --remove rootUrl
> > > > >> >
> > > > >> > This would get the KC client rep, change the corresponding
> fields
> > > and
> > > > >> send
> > > > >> > it back.
> > > > >> >
> > > > >> >
> > > > >> > >
> > > > >> > >
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>>
> > > > >> > >>> * Keycloak currently has a few problems with client
> > > registration and
> > > > >> > >>> it's necessary to modify the client before it will work
> > > correctly.
> > > > >> We
> > > > >> > >>> currently do this via the REST API. How are you planning on
> > > handling
> > > > >> > >>> these issues in your installer? It would be nice if the
> > > installer
> > > > >> was
> > > > >> > >>> aware of the Keycloak version and could apply "fix-ups" as
> > > needed
> > > > >> based
> > > > >> > >>> on the version.
> > > > >> > >>>
> > > > >> > >>
> > > > >> > >> AFAIK you have one problem? About not all redirect URI
> included
> > > from
> > > > >> the
> > > > >> > >> SAML entity descriptor. Is that what you are referring to or
> do
> > > you
> > > > >> have
> > > > >> > >> other problems?
> > > > >> > >>
> > > > >> > >> In either case fix-ups should be performed by the client
> > > registration
> > > > >> > >> services, not in the CLI.
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>>
> > > > >> > >>> * Keycloak has two ways to register a client (client
> > > registration
> > > > >> > >>> service vs. REST API). The two methods do not produce the
> same
> > > > >> client
> > > > >> > >>> configuration (I suspect because they do not share common
> code
> > > in
> > > > >> the
> > > > >> > >>> server). How are you planning on addressing the
> discrepancies?
> > > > >> > >>>
> > > > >> > >>
> > > > >> > >> The task of the CLI is not to address any discrepancies. It's
> > > just
> > > > >> > >> invoking the client reg services. Any discrepancies should be
> > > > >> handled by
> > > > >> > >> the client reg services themselves. Have you created JIRA's
> for
> > > > >> these or
> > > > >> > >> can you list them to us?
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>>
> > > > >> > >>> * The tool should be smart enough to produce a working
> client
> > > > >> without
> > > > >> > >>> manual intervention (i.e. the need to run admin cli commands
> > > > >> afterwards
> > > > >> > >>> to fix problems). Most admins won't know how to tweak the
> > > > >> configuration.
> > > > >> > >>>
> > > > >> > >>
> > > > >> > >> Can you list any you are aware of? Same comment as above
> applies
> > > > >> though,
> > > > >> > >> it's the responsibility of the client reg services to handle
> > > this,
> > > > >> not the
> > > > >> > >> CLI. Otherwise you'd have different behavior if you invoke
> > > client reg
> > > > >> > >> services directly rather than through the CLI.
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>>
> > > > >> > >>> * The tool should not have significant dependencies.
> > > > >> > >>>
> > > > >> > >>
> > > > >> > >> It'll be a fat-jar and will have a single dependency on the
> JVM.
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>>
> > > > >> > >>> Those are the thoughts off the top of my head, as you fill
> out
> > > the
> > > > >> > >>> details I'll continue to review. Recall the plan of record
> is
> > > for
> > > > >> > >>> Keycloak to provide such tools which we will then utilize.
> The
> > > > >> > >>> keycloak-httpd-client-install tool is a stop-gap solution
> until
> > > such
> > > > >> > >>> time as "offical" tools become available.
> > > > >> > >>>
> > > > >> > >>>
> > > > >> > >>>
> > > > >> > >>> --
> > > > >> > >>> John
> > > > >> > >>> _______________________________________________
> > > > >> > >>> keycloak-dev mailing list
> > > > >> > >>> keycloak-dev at lists.jboss.org
> > > > >> > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > > > >> > >>>
> > > > >> > >>
> > > > >> > >>
> > > > >> > >
> > > > >>
> > > > >> > _______________________________________________
> > > > >> > keycloak-dev mailing list
> > > > >> > keycloak-dev at lists.jboss.org
> > > > >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > > > >>
> > > > >>
> > > > >> --
> > > > >>
> > > > >> abstractj
> > > > >> PGP: 0x84DC9914
> > > > >>
> > > > >
> > > > >
> > >
> > > --
> > >
> > > abstractj
> > > PGP: 0x84DC9914
> > >
>
> --
>
> abstractj
> PGP: 0x84DC9914
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160630/a6c2819f/attachment-0001.html