[keycloak-dev] Client Registration CLI tool

Fri Jul 1 03:00:55 EDT 2016

On 30 June 2016 at 20:06, Bruno Oliveira <bruno at abstractj.org> wrote:

> Sorry about the duplicated e-mail.
>
> On Wed, Jun 29, 2016, 11:21 PM Stian Thorgersen <sthorger at redhat.com>
> wrote:
>
>> On 30 June 2016 at 08:08, Bruno Oliveira <bruno at abstractj.org> wrote:
>>
>>> What I suggested was based on what I saw at our current proposal:
>>>
>>> $ kcreg login -r master -u manage-client
>>> Server to connect to (http://localhost:8080/auth):
>>> Password:
>>>
>>> If you want to automate it for example in a shell script, unless I'm
>>> mistaken, you have to provide user's password or obtain initial access
>>> token through console. If we're assuming that's the default
>>> flow for automated scripts should be obtaining an initial access token
>>> from console first, that's fine.
>>>
>>
>> Yes, the password is just for manual use and wouldn't be cached or
>> anything like that.
>>
>>
>>>
>>> Flow for encryption:
>>>
>>> 1. Admin client retrieve the public key from the server
>>> 2. Admin client encrypt's whatever data is sensitive
>>> 3. Server decrypt's it
>>>
>>
>> Isn't that only going to hide the password? The encrypted password could
>> still be used as the server would accept/decrypt it.
>>
>
> Not only passwords, but username as well. Into other words any sensitive
> data.
>
> I don't follow you on this. The server will decrypt and validate those
> credentials.
>
> Anyways, this is just an idea, if it doesn't worth it, never mind.
>

I don't think it's needed in this case as we won't be storing
username/passwords. I'd still like to understand/talk about what you are
proposing though, but we should have that discussion in a separate thread ;)

>
>
>
>>
>>>
>>> On 2016-06-30, Stian Thorgersen wrote:
>>> > I can't see why we should consider it though:
>>> >
>>> > a) We have service account support, initial access tokens, offline
>>> tokens,
>>> > etc.. All designed so credentials don't need to be stored. Using a user
>>> > account directly is horrible as not only are you potentially exposing
>>> the
>>> > credentials, you're also giving all permissions of the user to the
>>> > automation. Using tokens or service account you can limit what
>>> permissions
>>> > the automation has.
>>> > b) If the credentials are encrypted, you'll also need to unlock the
>>> > vault/encrypted file. How would you suggest we do that? I can't see the
>>> > full flow here.
>>> >
>>> > On 30 June 2016 at 07:15, Bruno Oliveira <bruno at abstractj.org> wrote:
>>> >
>>> > > Yes, the encrypted data goes into public repository with
>>> .travis.yaml file.
>>> > > Only the Travis server is able to decrypt such data and validate
>>> it[1][2].
>>> > >
>>> > > Like I mentioned, not a blocker, but maybe something to take into
>>> > > consideration.
>>> > >
>>> > > [1] -
>>> > >
>>> https://docs.travis-ci.com/user/encryption-keys/#Notifications-Example
>>> > > [2] -
>>> https://github.com/twbs/bootstrap/blob/master/.travis.yml#L30-L32
>>> > >
>>> > > On 2016-06-30, Stian Thorgersen wrote:
>>> > > > The encryption support in Travis isn't that so you can encrypt
>>> sensitive
>>> > > > details that goes into .travis.yaml which is public through the
>>> repo?
>>> > > >
>>> > > > On 30 June 2016 at 06:52, Stian Thorgersen <sthorger at redhat.com>
>>> wrote:
>>> > > >
>>> > > > > I don't see the need for that. For the client registration CLI
>>> we will
>>> > > > > support initial access tokens as well as service accounts with
>>> > > pub/priv key
>>> > > > > authentication. For admin cli we'll support service accounts. No
>>> one
>>> > > should
>>> > > > > be using username/password combined with automated jobs.
>>> > > > >
>>> > > > > On 29 June 2016 at 21:49, Bruno Oliveira <bruno at abstractj.org>
>>> wrote:
>>> > > > >
>>> > > > >> I'm not sure if it's part of the scope for now. But
>>> > > > >> thinking about situations where you have to automate jobs plus
>>> provide
>>> > > > >> credentials without exposing them.
>>> > > > >>
>>> > > > >> My suggestion, even if it's not part of the scope for now is to
>>> > > encrypt
>>> > > > >> it,
>>> > > > >> like travis does[1]. I know that our plan is to deal of access
>>> token,
>>> > > > >> but would be nice to not expose credentials not even a single
>>> time.
>>> > > > >>
>>> > > > >>
>>> > > > >> [1] - https://docs.travis-ci.com/user/encryption-keys/
>>> > > > >>
>>> > > > >> On 2016-06-29, Stian Thorgersen wrote:
>>> > > > >> > On 28 June 2016 at 11:40, Marko Strukelj <mstrukel at redhat.com
>>> >
>>> > > wrote:
>>> > > > >> >
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > > On Tue, Jun 28, 2016 at 7:35 AM, Stian Thorgersen <
>>> > > > >> sthorger at redhat.com>
>>> > > > >> > > wrote:
>>> > > > >> > >
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >> On 27 June 2016 at 21:26, John Dennis <jdennis at redhat.com>
>>> > > wrote:
>>> > > > >> > >>
>>> > > > >> > >>> On 06/27/2016 07:48 AM, Marko Strukelj wrote:
>>> > > > >> > >>> > I've started work on Client Registration CLI tool. As a
>>> first
>>> > > > >> step,
>>> > > > >> > >>> here
>>> > > > >> > >>> > is a design document describing how I imagine the tool
>>> would
>>> > > be
>>> > > > >> used.
>>> > > > >> > >>> >
>>> > > > >> > >>> >
>>> > > > >> > >>> >
>>> > > > >> > >>>
>>> > > > >>
>>> > >
>>> https://docs.google.com/document/d/18SoZ34sY_k7N8ae-WDsvo7QeI-cHkpTURIlUk5dpIhU/edit?usp=sharing
>>> > > > >> > >>> >
>>> > > > >> > >>> >
>>> > > > >> > >>> > I'll use this document as a spec / guide as I implement
>>> the
>>> > > client
>>> > > > >> > >>> tool.
>>> > > > >> > >>> >
>>> > > > >> > >>> > Within days I'll also send a link to initial ideas for
>>> Admin
>>> > > > >> Client
>>> > > > >> > >>> tool
>>> > > > >> > >>> > which in principle should allow administrator to
>>> configure
>>> > > > >> everything
>>> > > > >> > >>> > that can otherwise be done through Admin Console.
>>> > > > >> > >>> >
>>> > > > >> > >>> > Any feedback welcome.
>>> > > > >> > >>>
>>> > > > >> > >>> FWIW we've already written a client registration tool for
>>> > > Keycloak.
>>> > > > >> At
>>> > > > >> > >>> the moment it is specifically targeted for SAML clients
>>> (SP,
>>> > > Service
>>> > > > >> > >>> Provider) in Apache HTTPD but we have plans to extend it
>>> to
>>> > > OIDC.
>>> > > > >> > >>>
>>> > > > >> > >>> It is currently in Fedora and will also ship in OSP.
>>> > > > >> > >>>
>>> > > > >> > >>> It is hosted here:
>>> > > > >> > >>> https://github.com/jdennis/keycloak-httpd-client-install
>>> > > > >> > >>>
>>> > > > >> > >>> The man page for it (formatted for HTML) can be found
>>> here:
>>> > > > >> > >>>
>>> > > > >>
>>> > >
>>> https://jdennis.fedorapeople.org/doc/keycloak-httpd-client-install.html
>>> > > > >> > >>>
>>> > > > >> > >>> The man page discusses 3 different ways you can
>>> authenticate
>>> > > and 2
>>> > > > >> > >>> different ways client registration can be performed.
>>> > > > >> > >>>
>>> > > > >> > >>> I have a lot of experience with Keycloak client
>>> registration
>>> > > tools
>>> > > > >> and
>>> > > > >> > >>> have worked through many issues, I'm happy to share my
>>> > > experience.
>>> > > > >> > >>>
>>> > > > >> > >>> Here are some thoughts/issues you may want to take into
>>> account:
>>> > > > >> > >>>
>>> > > > >> > >>> * The tool must be capable of running without
>>> interactivity as
>>> > > part
>>> > > > >> of a
>>> > > > >> > >>> scripted installation task.
>>> > > > >> > >>>
>>> > > > >> > >>> * It should not depend on a home directory being
>>> available.
>>> > > > >> > >>>
>>> > > > >> > >>> * If a home directory is utilized how will you
>>> disambiguate any
>>> > > > >> stored
>>> > > > >> > >>> state belonging to a script that is run by different
>>> processes
>>> > > but
>>> > > > >> under
>>> > > > >> > >>> the same user (possibly simultaneously)? To clarify, many
>>> > > install
>>> > > > >> tools
>>> > > > >> > >>> run as the root user or some other admin user. Each
>>> invocation
>>> > > of
>>> > > > >> these
>>> > > > >> > >>> install tools can be run with entirely different
>>> parameters and
>>> > > may
>>> > > > >> > >>> execute either in parallel or partially overlapping in
>>> time.
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > > Maybe I should have included this link in the design
>>> document to
>>> > > make
>>> > > > >> it
>>> > > > >> > > clear to everyone what Client Registration this tool is for:
>>> > > > >> > >
>>> > > > >>
>>> > >
>>> http://keycloak.github.io/docs/userguide/keycloak-server/html/client-registration.html
>>> > > > >> > >
>>> > > > >> > > It's a REST API defined by specs, and is separate from
>>> Admin REST
>>> > > API.
>>> > > > >> > >
>>> > > > >> > > About using home directory, the way I see it - you either a)
>>> > > specify
>>> > > > >> all
>>> > > > >> > > the state when executing a command, or b) you have a
>>> mechanism
>>> > > that
>>> > > > >> allows
>>> > > > >> > > the concept of 'session' between command invocations.
>>> > > > >> > >
>>> > > > >> > > If you use the first approach (a) then on each invocation
>>> of the
>>> > > > >> command
>>> > > > >> > > you have to specify either username:password, or a token.
>>> The
>>> > > client
>>> > > > >> > > registration specification defines workflow for Initial
>>> Access
>>> > > > >> Tokens, and
>>> > > > >> > > Registration Access Tokens, which require to automatically
>>> > > intercept a
>>> > > > >> > > newly issued token after each CRUD operation, and save it
>>> for any
>>> > > > >> > > subsequent operation on the same client resource. I can't
>>> see how
>>> > > this
>>> > > > >> > > could be achieved by using the first approach.
>>> > > > >> > >
>>> > > > >> > > For the second approach (b) you need a way to communicate
>>> > > 'session'
>>> > > > >> state.
>>> > > > >> > > The state we are saving are just tokens associated with
>>> current
>>> > > user
>>> > > > >> or
>>> > > > >> > > specific clients, or specific grants. Looks to me that if
>>> multiple
>>> > > > >> parallel
>>> > > > >> > > sessions are in collision about these tokens then the cli
>>> tool
>>> > > itself
>>> > > > >> might
>>> > > > >> > > be used the wrong way. Namely, once the client
>>> authenticates with
>>> > > a
>>> > > > >> login,
>>> > > > >> > > access token and refresh token are cached. Multiple client
>>> > > instances
>>> > > > >> can
>>> > > > >> > > use the same access token, and the same refresh token. A
>>> thing to
>>> > > > >> maybe be
>>> > > > >> > > careful about is just properly locking the file when making
>>> > > changes
>>> > > > >> to it.
>>> > > > >> > > For initial access token you have to explicitly add it, and
>>> > > assign it
>>> > > > >> an
>>> > > > >> > > alias - you can use any random value there if you want. For
>>> > > > >> registration
>>> > > > >> > > access token they are automatically associated with initial
>>> token
>>> > > > >> they were
>>> > > > >> > > initiated from - again there should be no collision.
>>> > > > >> > >
>>> > > > >> >
>>> > > > >> > I like the option to have two approaches as there are two
>>> use-cases.
>>> > > > >> One is
>>> > > > >> > manually registering for example during development or when
>>> manually
>>> > > > >> > configuring an application to use Keycloak. Another is
>>> scripted.
>>> > > Even
>>> > > > >> for
>>> > > > >> > scripted you may quite likely want to just add service account
>>> > > > >> credentials
>>> > > > >> > or initial access token directly to ~/.keycloak/ rather than
>>> pass
>>> > > these
>>> > > > >> to
>>> > > > >> > the commands.
>>> > > > >> >
>>> > > > >> > Registration access tokens are associated with a client, not
>>> an
>>> > > initial
>>> > > > >> > access token. Also, remember the registration access token is
>>> > > changed on
>>> > > > >> > updates.
>>> > > > >> >
>>> > > > >> >
>>> > > > >> > >
>>> > > > >> > > What alternative mechanism would you suggest for storing
>>> 'session'
>>> > > > >> info?
>>> > > > >> > > We want to support Windows as well so it can't be Unix /
>>> Bash
>>> > > > >> specific.
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > >>
>>> > > > >> > >>> * The tool should be idempotent.
>>> > > > >> > >>>
>>> > > > >> > >>> * You suggest storing tokens in a cache, how do you plan
>>> on
>>> > > > >> handling the
>>> > > > >> > >>> case where a token expires before all operations are
>>> complete?
>>> > > > >> > >>>
>>> > > > >> > >>> * We also initially took the approach of caching tokens
>>> but
>>> > > > >> discovered
>>> > > > >> > >>> the complexity did not justify the minimal cost of
>>> obtaining a
>>> > > new
>>> > > > >> token
>>> > > > >> > >>> for each invocation. This greatly simplified the code
>>> with very
>>> > > > >> little
>>> > > > >> > >>> performance impact.
>>> > > > >> > >>>
>>> > > > >> > >>> * You do not mention what type of client you're
>>> registering. I'm
>>> > > > >> > >>> assuming it's OpenID but SAML clients (SP) are equally
>>> > > important.
>>> > > > >> The
>>> > > > >> > >>> tool must be able to handle both.
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > >> Marko is probably referring to the Keycloak client
>>> > > representation,
>>> > > > >> which
>>> > > > >> > >> can be either OpenID or SAML. However, we also need to
>>> support
>>> > > OpenID
>>> > > > >> > >> Connect client descriptions as well as SAML entity
>>> descriptors as
>>> > > > >> both are
>>> > > > >> > >> supported by client reg services.
>>> > > > >> > >>
>>> > > > >> > >
>>> > > > >> > > The CLI needs to know which of the client registration
>>> providers
>>> > > (REST
>>> > > > >> > > endpoints) to use - there are four as described in the
>>> Client
>>> > > > >> Registration
>>> > > > >> > > documentation (
>>> > > > >> > >
>>> > > > >>
>>> > >
>>> http://keycloak.github.io/docs/userguide/keycloak-server/html/client-registration.html
>>> > > > >> > > )
>>> > > > >> > >
>>> > > > >> > > Ideally the input format of the file could be recognised as
>>> only
>>> > > > >> > > appropriate for one of these providers, and the correct
>>> provider
>>> > > then
>>> > > > >> > > automatically used. But maybe we need a way to explicitly
>>> tell the
>>> > > > >> tool
>>> > > > >> > > what provider to use. For example:
>>> > > > >> > >
>>> > > > >> > > kc new --type default --name test-app --enabled true
>>> --base-url
>>> > > > >> > > http://localhost:8480/test-app --redirect-uri '
>>> > > > >> > > http://localhost:8480/test-app/*' --admin-url
>>> > > > >> > > http://localhost:8480/test-app/logout --secret password |
>>> kc
>>> > > create
>>> > > > >> > > --type default -f -
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > > Having to set --type for both creating a description (kc
>>> new), and
>>> > > > >> > > pushing it to the server (kc create) is not ideal.
>>> > > > >> > >
>>> > > > >> >
>>> > > > >> > We can detect the difference between Keycloak client rep,
>>> SAML and
>>> > > OIDC
>>> > > > >> > json that's pretty trivial and we should do that. However,
>>> there
>>> > > needs
>>> > > > >> to
>>> > > > >> > be a way to specify the provider as well as there could be
>>> custom
>>> > > > >> providers
>>> > > > >> > added.
>>> > > > >> >
>>> > > > >> > For getting the client it should by default return Keycloak
>>> client
>>> > > > >> > representation, but we need an option to be able to specify
>>> the
>>> > > > >> provider so
>>> > > > >> > it can return the client installation file instead.
>>> > > > >> >
>>> > > > >> > With regards to creating by passing arguments rather than a
>>> file
>>> > > that
>>> > > > >> > should only support client representation files.
>>> > > > >> >
>>> > > > >> >
>>> > > > >> >
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >>>
>>> > > > >> > >>> * I don't see anything in your document on how to specify
>>> the
>>> > > SAML
>>> > > > >> > >>> metadata.
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > > Instead of piping in my-client.json, you would pipe in
>>> > > > >> my-client-saml.xml,
>>> > > > >> > > possibly requiring an extra --type specifier as described
>>> above.
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > >>
>>> > > > >> > >>> * I don't see anything in your document on how the user
>>> > > modifies the
>>> > > > >> > >>> client. It appears as if you are retrieving a
>>> > > ClientRepresentation
>>> > > > >> JSON
>>> > > > >> > >>> document and expecting the user to edit it in a text
>>> editor
>>> > > which
>>> > > > >> will
>>> > > > >> > >>> then be sent back. That won't work for non-interactive
>>> > > installs. It
>>> > > > >> also
>>> > > > >> > >>> presumes the user knows how to read and modify the JSON.
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > >> It would be nice to be able to set specific fields without
>>> > > having to
>>> > > > >> > >> modify JSON. We discussed that for the Admin CLI, but we
>>> should
>>> > > > >> probably
>>> > > > >> > >> also add it to the client reg CLI
>>> > > > >> > >>
>>> > > > >> > >
>>> > > > >> > > It would be very valuable to see some of the usecases, what
>>> kind
>>> > > of
>>> > > > >> > > changes people do on existing clients.
>>> > > > >> > >
>>> > > > >> >
>>> > > > >> > We should support anything that is in client representation.
>>> It
>>> > > should
>>> > > > >> just
>>> > > > >> > be a generic way to change json fields. For example:
>>> > > > >> >
>>> > > > >> > # kcreg update <client id> --set enabled=false --set baseUrl=
>>> > > > >> > http://new-url/myapp --remove rootUrl
>>> > > > >> >
>>> > > > >> > This would get the KC client rep, change the corresponding
>>> fields
>>> > > and
>>> > > > >> send
>>> > > > >> > it back.
>>> > > > >> >
>>> > > > >> >
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >>>
>>> > > > >> > >>> * Keycloak currently has a few problems with client
>>> > > registration and
>>> > > > >> > >>> it's necessary to modify the client before it will work
>>> > > correctly.
>>> > > > >> We
>>> > > > >> > >>> currently do this via the REST API. How are you planning
>>> on
>>> > > handling
>>> > > > >> > >>> these issues in your installer? It would be nice if the
>>> > > installer
>>> > > > >> was
>>> > > > >> > >>> aware of the Keycloak version and could apply "fix-ups" as
>>> > > needed
>>> > > > >> based
>>> > > > >> > >>> on the version.
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > >> AFAIK you have one problem? About not all redirect URI
>>> included
>>> > > from
>>> > > > >> the
>>> > > > >> > >> SAML entity descriptor. Is that what you are referring to
>>> or do
>>> > > you
>>> > > > >> have
>>> > > > >> > >> other problems?
>>> > > > >> > >>
>>> > > > >> > >> In either case fix-ups should be performed by the client
>>> > > registration
>>> > > > >> > >> services, not in the CLI.
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >>>
>>> > > > >> > >>> * Keycloak has two ways to register a client (client
>>> > > registration
>>> > > > >> > >>> service vs. REST API). The two methods do not produce the
>>> same
>>> > > > >> client
>>> > > > >> > >>> configuration (I suspect because they do not share common
>>> code
>>> > > in
>>> > > > >> the
>>> > > > >> > >>> server). How are you planning on addressing the
>>> discrepancies?
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > >> The task of the CLI is not to address any discrepancies.
>>> It's
>>> > > just
>>> > > > >> > >> invoking the client reg services. Any discrepancies should
>>> be
>>> > > > >> handled by
>>> > > > >> > >> the client reg services themselves. Have you created
>>> JIRA's for
>>> > > > >> these or
>>> > > > >> > >> can you list them to us?
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >>>
>>> > > > >> > >>> * The tool should be smart enough to produce a working
>>> client
>>> > > > >> without
>>> > > > >> > >>> manual intervention (i.e. the need to run admin cli
>>> commands
>>> > > > >> afterwards
>>> > > > >> > >>> to fix problems). Most admins won't know how to tweak the
>>> > > > >> configuration.
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > >> Can you list any you are aware of? Same comment as above
>>> applies
>>> > > > >> though,
>>> > > > >> > >> it's the responsibility of the client reg services to
>>> handle
>>> > > this,
>>> > > > >> not the
>>> > > > >> > >> CLI. Otherwise you'd have different behavior if you invoke
>>> > > client reg
>>> > > > >> > >> services directly rather than through the CLI.
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >>>
>>> > > > >> > >>> * The tool should not have significant dependencies.
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > >> It'll be a fat-jar and will have a single dependency on
>>> the JVM.
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >>>
>>> > > > >> > >>> Those are the thoughts off the top of my head, as you
>>> fill out
>>> > > the
>>> > > > >> > >>> details I'll continue to review. Recall the plan of
>>> record is
>>> > > for
>>> > > > >> > >>> Keycloak to provide such tools which we will then
>>> utilize. The
>>> > > > >> > >>> keycloak-httpd-client-install tool is a stop-gap solution
>>> until
>>> > > such
>>> > > > >> > >>> time as "offical" tools become available.
>>> > > > >> > >>>
>>> > > > >> > >>>
>>> > > > >> > >>>
>>> > > > >> > >>> --
>>> > > > >> > >>> John
>>> > > > >> > >>> _______________________________________________
>>> > > > >> > >>> keycloak-dev mailing list
>>> > > > >> > >>> keycloak-dev at lists.jboss.org
>>> > > > >> > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> > > > >> > >>>
>>> > > > >> > >>
>>> > > > >> > >>
>>> > > > >> > >
>>> > > > >>
>>> > > > >> > _______________________________________________
>>> > > > >> > keycloak-dev mailing list
>>> > > > >> > keycloak-dev at lists.jboss.org
>>> > > > >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> > > > >>
>>> > > > >>
>>> > > > >> --
>>> > > > >>
>>> > > > >> abstractj
>>> > > > >> PGP: 0x84DC9914
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > >
>>> > > --
>>> > >
>>> > > abstractj
>>> > > PGP: 0x84DC9914
>>> > >
>>>
>>> --
>>>
>>> abstractj
>>> PGP: 0x84DC9914
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160701/13646359/attachment-0001.html 