From mposolda at redhat.com Wed Aug 1 08:43:59 2018 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 1 Aug 2018 14:43:59 +0200 Subject: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication In-Reply-To: References: Message-ID: On 30/07/18 09:20, Sebastien Blanc wrote: > Thanks Takashi, this is really useful information. Some comments inline > > On Mon, Jul 30, 2018 at 2:40 AM, ???? / NORIMATSU?TAKASHI < > takashi.norimatsu.ws at hitachi.com> wrote: > >> Hello Sebastien, >> >> I've read the ticket you mentioned: https://issues.jboss.org/ >> browse/KEYCLOAK-7635 >> >> I'm not sure what the token review endpoint is (similar to RFC 7662 OAuth >> 2.0 Token Introspection?), but I inferred the following situation. >> * OpenShift accesses keycloak's token review endpoint with some token, but >> without client_id query parameter or client_id encoded Authorization >> header. Therefore, keycloak cannot conduct client authentication. In this >> situation, a client certificate might be used for client authentication in >> the context of OAuth 2.0. >> >> Also, I'm not sure how to enclose client_id or client_id identifiable >> information onto a client certificate and extract it. So I would like to >> comment on other aspects other that it. >> >> - register client dn >> According to 2.1.2. Client Registration Metadata of OAuth 2.0 Mutual TLS >> Client Authentication and Certificate Bound Access Tokens ( >> https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1.2), each >> client has "tls_client_auth_subject_dn" metadata parameter's value. This >> has already been mentioned by @mposolda in https://issues.jboss.org/ >> browse/KEYCLOAK-7512 >> >> In keycloak, OIDCClientRepresentation defines these OIDC Dynamic client >> registration properties. >> >> OIDCAdvancedConfigWrapper can be used to convert these properties onto >> attributes of the client model. >> >> DescriptionConverter can be used to convert internal representation of the >> client onto external one (through OIDC Dynamic client registration) and >> vice versa. >> >> IMO, at least, those three classes have to be modified to support this new >> OIDC Client Registration property "tls_client_auth_subject_dn". >> > This is a nice complement to what Marek said in the ticket and makes thing > clearer for me as well. > But this would mean that Openshift should use dynamic client registration > to make this flow works ? Now I remember that few months ago, Bill implemented the support for ClientStorage SPI. And I think he implemented it mainly because of Openshift. It somehow allows to provision clients automatically from 3rd party source (EG. from Openshift) without a need to manually register/create them. So I guess (not 100% sure) that Openshift integration will use this for client provisioning and it won't use dynamic client registration. IMO using dynamic client registration will be more aligned with the OIDC specs, but I guess we don't have a "power" to change the behaviour of Openshift. Or do we? :-) But support for the OIDC Dynamic client registration, which Takashi mentions, and which is mentioned in the specification, will be good to have IMO even if it's not something directly useful for Openshift. If it's not priority for you and Openshift integration, we can maybe at least create some follow-up JIRA to add it later? Marek > >> - extract client certificate >> >> X509ClientCertificateLookup : >> https://github.com/keycloak/keycloak/blob/master/services/ >> src/main/java/org/keycloak/services/x509/X509ClientCertificateLookup.java >> This Client Certificate Lookup provider can be used to leverage the >> feature supporting reverse proxy deployed environment mentioned in the >> server admin manual (https://www.keycloak.org/docs/latest/server_admin/ >> index.html#client-certificate-lookup) . >> >> You can consult >> https://github.com/keycloak/keycloak/blob/master/services/ >> src/main/java/org/keycloak/authentication/authenticators/x509/ >> AbstractX509ClientCertificateAuthenticator.java#L196 >> to get how to use this client certificate lookup feature. >> >> In order to use it, KeycloakSession and HttpRequest instances are required >> to find the Client Certificate Lookup provider implementation. >> >> I've found that current ClientAuthenticator can get it by >> ClientAuthenticationFlowContext.getSession() and .getHttpRequest() in >> ClientAuthenticator.authenticateClient(ClientAuthenticationFlowContext >> context). >> > Yes, I started to test that, exactly with this classes and methods but > until now I fail to retrieve any certificate (probably because of my local > setup/configuration). > >> - server side metadata advertisement >> >> OIDCConfigurationRepresentation is a server side metadata representation >> and might be related to 2.1.1. PKI Authentication Method Metadata Value of >> OAuth 2.0 Mutual TLS Client Authentication and Certificate Bound Access >> Tokens (https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1.1 >> ) >> >> Best regards, >> Takashi Norimatsu >> Hitachi Ltd., >> >> ---------- >> From: Sebastien Blanc >> Sent: Friday, July 27, 2018 4:31 PM >> To: ???? / NORIMATSU?TAKASHI >> Cc: Sebastian Laskawiec ; >> keycloak-dev at lists.jboss.org >> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication >> >> Hi Takashi ! >> >> You can even help before if you want to :) >> >> The ticket is here : https://issues.jboss.org/browse/KEYCLOAK-7635 >> >> I created an "empty" X509ClientAuthenticator" branch here : >> https://github.com/sebastienblanc/keycloak/tree/client-x509 >> >> And I'm investigating what we can reuse from this pacakge : >> https://github.com/keycloak/keycloak/tree/master/services/ >> src/main/java/org/keycloak/authentication/authenticators/x509 >> >> Any feedback, help, advise is welcome ! >> >> Sebi >> >> >> On Fri, Jul 27, 2018 at 3:22 AM, ???? / NORIMATSU?TAKASHI < >> takashi.norimatsu.ws at hitachi.com> wrote: >> >>> Hello Sebastian, >>> >>> I'm looking forward to your work, and I would be happy if I could make >>> some contribution after finishing your work. >>> >>> Best regards, >>> Takashi Norimatsu >>> Hitachi Ltd., >>> >>> ---------- >>> From: Sebastian Laskawiec >>> Sent: Thursday, July 26, 2018 5:24 PM >>> To: ???? / NORIMATSU?TAKASHI >>> Cc: keycloak-dev at lists.jboss.org >>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication >>> >>> Hey Takashi, >>> >>> Thanks a lot for the interest in contributing Keycloak! >>> >>> Sebi and I are working on this topic currently. We plan to reuse some >> bits >>> of the User x509 Authentication and bring them to the client. We planned >>> the implementation for this sprint, so it *should* be ready in ~3 weeks. >>> >>> More comments inlined. >>> >>> Thanks, >>> Sebastian >>> On Thu, Jul 26, 2018 at 1:23 AM ???? / NORIMATSU?TAKASHI < >>> takashi.norimatsu.ws at hitachi.com> wrote: >>> Hello, >>> >>> As for mentioned in https://issues.jboss.org/browse/KEYCLOAK-7512 and >>> https://issues.jboss.org/browse/KEYCLOAK-7635, Is there anyone who >>> currently implements OAuth 2.0 Mutual TLS Client Authentication defined >> in >>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 ? >>> >>> We also have additional requirement - allow to authenticate client >> without >>> "client_id" being sent (we need to extract it from the Certificate >> obtained >>> during TLS Handshake). This is required for OpenShift integration. >>> >>> >>> If no one does it, I would like to try to implement this feature. What do >>> you think about it ? >>> >>> Also, In https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2, >>> two types of OAuth 2.0 Mutual TLS Client Authentication are defined, for >>> PKI and for Self-Signed Certificate. >>> >>> I would be happy if you who are interested in this feature tell me which >>> you like better. >>> >>> As far as I know, we won't be touching self-registering clients. So maybe >>> once we are done (let's assume that will happen in ~3 weeks), you could >>> take it over and look into that? >>> >>> BTW, as for now, we will be implementing everything in this branch: >>> https://github.com/sebastienblanc/keycloak/tree/client-x509 (currently, >>> it contains an empty Authenticator but we will be adding bits and pieces >> to >>> it). >>> >>> >>> Best regards, >>> Takashi Norimatsu >>> Hitachi Ltd., >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From mposolda at redhat.com Wed Aug 1 09:47:03 2018 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 1 Aug 2018 15:47:03 +0200 Subject: [keycloak-dev] Keycloak 4.2.0.Final released! Message-ID: <222f27b6-087f-0ffb-40e4-745b06cb8f01@redhat.com> See details on the blog: http://blog.keycloak.org/2018/08/keycloak-420final-released.html From dt at acutus.pro Thu Aug 2 06:40:51 2018 From: dt at acutus.pro (Dmitry Telegin) Date: Thu, 02 Aug 2018 13:40:51 +0300 Subject: [keycloak-dev] Keycloak and SAML AudienceRestriction Message-ID: <1533206451.6676.1.camel@acutus.pro> Hi, It's been SAML time recently in keycloak-dev, so I won't be breaking the trend... :) A customer tasked us with configuring Keycloak brokering to the 3rd party SAML IdP. The IdP doesn't allow for SP metadata import, so the values have to be configured manually, of which the two are mandatory, namely Assertion Consumer Service URL and Audience (Entity ID). While things are crystal clear with ACS URL, there was some misunderstanding with the Audience parameter. Assuming that it should be equal to the EntityID of Keycloak (acting as an SP in this case), we've put it there. After that, while reconfiguring for IdP-initiated SSO, we have changed the ACS (the /clients/{url-name} suffix is appended to it), but the question was what to do with Entity ID. By experiment, we have determined that actually any non-empty value worked. The situation is ambiguous, and we need to communicate it to the customer somehow. The line in the docs "put any non-empty value" smells fishy to me. I've found a technical explanation though; the Audience (Entity ID) value ends up in the AudienceRestriction tag of the SAML response. While Keycloak's SAML parser is aware of that tag, it isn't processed in any way (ignored, in other words). Here's what the SAML spec says on AudienceRestriction: > ?Although a SAML relying party that is outside the audiences > specified is capable of drawing conclusions from an assertion, the > SAML asserting party explicitly makes no representation as to > accuracy or trustworthiness to such a party... > > ??...the element allows the SAML asserting > party to state explicitly that no warranty is provided to such a > party in a machine- and human-readable form. While there can be no > guarantee that a court would uphold such a warrantyexclusion in every > circumstance, the probability of upholding the warranty exclusion is > considerably improved... http://docs.oasis-open.org/security/saml/v2.0/saml-core-2.0-os.pdf Nothing is said in the spec about if the AudienceRestriction check is mandatory, so I'd suppose it is optional. Some SAML-enabled software however implements strict checking, WebLogic being a well-known case. So it doesn't look like a defect or a security vulnerability, and shouldn't pose any problems? Wanted to know the stance of the Keycloak dev team on this. Thanks in advance! Dmitry Telegin CTO, Acutus s.r.o. Keycloak Consulting and Training Pod lipami street 339/52, 130 00 Prague 3, Czech Republic +42 (022) 888-30-71 E-mail: info at acutus.pro From sblanc at redhat.com Thu Aug 2 08:41:59 2018 From: sblanc at redhat.com (Sebastien Blanc) Date: Thu, 2 Aug 2018 14:41:59 +0200 Subject: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication In-Reply-To: References: Message-ID: So the first part , Client x509 Authentication is available here for review https://github.com/keycloak/keycloak/pull/5435 , the client_id will be lookup on the form data and the query parameters. And as Marek said I would like to follow up with another ticket for support of OIDC Dynamic Client Registration and the support of the `tls_client_auth_subject_dn`. Seb On Wed, Aug 1, 2018 at 2:43 PM, Marek Posolda wrote: > On 30/07/18 09:20, Sebastien Blanc wrote: > >> Thanks Takashi, this is really useful information. Some comments inline >> >> On Mon, Jul 30, 2018 at 2:40 AM, ???? / NORIMATSU?TAKASHI < >> takashi.norimatsu.ws at hitachi.com> wrote: >> >> Hello Sebastien, >>> >>> I've read the ticket you mentioned: https://issues.jboss.org/ >>> browse/KEYCLOAK-7635 >>> >>> I'm not sure what the token review endpoint is (similar to RFC 7662 OAuth >>> 2.0 Token Introspection?), but I inferred the following situation. >>> * OpenShift accesses keycloak's token review endpoint with some token, >>> but >>> without client_id query parameter or client_id encoded Authorization >>> header. Therefore, keycloak cannot conduct client authentication. In this >>> situation, a client certificate might be used for client authentication >>> in >>> the context of OAuth 2.0. >>> >>> Also, I'm not sure how to enclose client_id or client_id identifiable >>> information onto a client certificate and extract it. So I would like to >>> comment on other aspects other that it. >>> >>> - register client dn >>> According to 2.1.2. Client Registration Metadata of OAuth 2.0 Mutual TLS >>> Client Authentication and Certificate Bound Access Tokens ( >>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1.2), >>> each >>> client has "tls_client_auth_subject_dn" metadata parameter's value. This >>> has already been mentioned by @mposolda in https://issues.jboss.org/ >>> browse/KEYCLOAK-7512 >>> >>> In keycloak, OIDCClientRepresentation defines these OIDC Dynamic client >>> registration properties. >>> >>> OIDCAdvancedConfigWrapper can be used to convert these properties onto >>> attributes of the client model. >>> >>> DescriptionConverter can be used to convert internal representation of >>> the >>> client onto external one (through OIDC Dynamic client registration) and >>> vice versa. >>> >>> IMO, at least, those three classes have to be modified to support this >>> new >>> OIDC Client Registration property "tls_client_auth_subject_dn". >>> >>> This is a nice complement to what Marek said in the ticket and makes >> thing >> clearer for me as well. >> But this would mean that Openshift should use dynamic client registration >> to make this flow works ? >> > > Now I remember that few months ago, Bill implemented the support for > ClientStorage SPI. And I think he implemented it mainly because of > Openshift. It somehow allows to provision clients automatically from 3rd > party source (EG. from Openshift) without a need to manually > register/create them. So I guess (not 100% sure) that Openshift integration > will use this for client provisioning and it won't use dynamic client > registration. > > IMO using dynamic client registration will be more aligned with the OIDC > specs, but I guess we don't have a "power" to change the behaviour of > Openshift. Or do we? :-) > > But support for the OIDC Dynamic client registration, which Takashi > mentions, and which is mentioned in the specification, will be good to have > IMO even if it's not something directly useful for Openshift. If it's not > priority for you and Openshift integration, we can maybe at least create > some follow-up JIRA to add it later? > > Marek > >> >> - extract client certificate >>> >>> X509ClientCertificateLookup : >>> https://github.com/keycloak/keycloak/blob/master/services/ >>> src/main/java/org/keycloak/services/x509/X509ClientCertifica >>> teLookup.java >>> This Client Certificate Lookup provider can be used to leverage the >>> feature supporting reverse proxy deployed environment mentioned in the >>> server admin manual (https://www.keycloak.org/docs/latest/server_admin/ >>> index.html#client-certificate-lookup) . >>> >>> You can consult >>> https://github.com/keycloak/keycloak/blob/master/services/ >>> src/main/java/org/keycloak/authentication/authenticators/x509/ >>> AbstractX509ClientCertificateAuthenticator.java#L196 >>> to get how to use this client certificate lookup feature. >>> >>> In order to use it, KeycloakSession and HttpRequest instances are >>> required >>> to find the Client Certificate Lookup provider implementation. >>> >>> I've found that current ClientAuthenticator can get it by >>> ClientAuthenticationFlowContext.getSession() and .getHttpRequest() in >>> ClientAuthenticator.authenticateClient(ClientAuthenticationFlowContext >>> context). >>> >>> Yes, I started to test that, exactly with this classes and methods but >> until now I fail to retrieve any certificate (probably because of my local >> setup/configuration). >> >> - server side metadata advertisement >>> >>> OIDCConfigurationRepresentation is a server side metadata representation >>> and might be related to 2.1.1. PKI Authentication Method Metadata Value >>> of >>> OAuth 2.0 Mutual TLS Client Authentication and Certificate Bound Access >>> Tokens (https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#sectio >>> n-2.1.1 >>> ) >>> >>> Best regards, >>> Takashi Norimatsu >>> Hitachi Ltd., >>> >>> ---------- >>> From: Sebastien Blanc >>> Sent: Friday, July 27, 2018 4:31 PM >>> To: ???? / NORIMATSU?TAKASHI >>> Cc: Sebastian Laskawiec ; >>> keycloak-dev at lists.jboss.org >>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication >>> >>> Hi Takashi ! >>> >>> You can even help before if you want to :) >>> >>> The ticket is here : https://issues.jboss.org/browse/KEYCLOAK-7635 >>> >>> I created an "empty" X509ClientAuthenticator" branch here : >>> https://github.com/sebastienblanc/keycloak/tree/client-x509 >>> >>> And I'm investigating what we can reuse from this pacakge : >>> https://github.com/keycloak/keycloak/tree/master/services/ >>> src/main/java/org/keycloak/authentication/authenticators/x509 >>> >>> Any feedback, help, advise is welcome ! >>> >>> Sebi >>> >>> >>> On Fri, Jul 27, 2018 at 3:22 AM, ???? / NORIMATSU?TAKASHI < >>> takashi.norimatsu.ws at hitachi.com> wrote: >>> >>> Hello Sebastian, >>>> >>>> I'm looking forward to your work, and I would be happy if I could make >>>> some contribution after finishing your work. >>>> >>>> Best regards, >>>> Takashi Norimatsu >>>> Hitachi Ltd., >>>> >>>> ---------- >>>> From: Sebastian Laskawiec >>>> Sent: Thursday, July 26, 2018 5:24 PM >>>> To: ???? / NORIMATSU?TAKASHI >>>> Cc: keycloak-dev at lists.jboss.org >>>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client >>>> Authentication >>>> >>>> Hey Takashi, >>>> >>>> Thanks a lot for the interest in contributing Keycloak! >>>> >>>> Sebi and I are working on this topic currently. We plan to reuse some >>>> >>> bits >>> >>>> of the User x509 Authentication and bring them to the client. We planned >>>> the implementation for this sprint, so it *should* be ready in ~3 weeks. >>>> >>>> More comments inlined. >>>> >>>> Thanks, >>>> Sebastian >>>> On Thu, Jul 26, 2018 at 1:23 AM ???? / NORIMATSU?TAKASHI < >>>> takashi.norimatsu.ws at hitachi.com> wrote: >>>> Hello, >>>> >>>> As for mentioned in https://issues.jboss.org/browse/KEYCLOAK-7512 and >>>> https://issues.jboss.org/browse/KEYCLOAK-7635, Is there anyone who >>>> currently implements OAuth 2.0 Mutual TLS Client Authentication defined >>>> >>> in >>> >>>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 ? >>>> >>>> We also have additional requirement - allow to authenticate client >>>> >>> without >>> >>>> "client_id" being sent (we need to extract it from the Certificate >>>> >>> obtained >>> >>>> during TLS Handshake). This is required for OpenShift integration. >>>> >>>> >>>> If no one does it, I would like to try to implement this feature. What >>>> do >>>> you think about it ? >>>> >>>> Also, In https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 >>>> , >>>> two types of OAuth 2.0 Mutual TLS Client Authentication are defined, for >>>> PKI and for Self-Signed Certificate. >>>> >>>> I would be happy if you who are interested in this feature tell me which >>>> you like better. >>>> >>>> As far as I know, we won't be touching self-registering clients. So >>>> maybe >>>> once we are done (let's assume that will happen in ~3 weeks), you could >>>> take it over and look into that? >>>> >>>> BTW, as for now, we will be implementing everything in this branch: >>>> https://github.com/sebastienblanc/keycloak/tree/client-x509 (currently, >>>> it contains an empty Authenticator but we will be adding bits and pieces >>>> >>> to >>> >>>> it). >>>> >>>> >>>> Best regards, >>>> Takashi Norimatsu >>>> Hitachi Ltd., >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > From mposolda at redhat.com Fri Aug 3 02:30:58 2018 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 3 Aug 2018 08:30:58 +0200 Subject: [keycloak-dev] Keycloak 4.2.1.Final released Message-ID: This release just fixes some DB migration related bugs. See details on the blog: http://blog.keycloak.org/2018/08/keycloak-421final-released.html From slaskawi at redhat.com Mon Aug 6 05:11:54 2018 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 6 Aug 2018 11:11:54 +0200 Subject: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication In-Reply-To: References: Message-ID: Hey Takashi, As the initial version is more or less done, I created a follow-up JIRA with dynamic client registration and Mutual TLS [1]. Perhaps you'd be interested in looking into it? Thanks, Sebastian [1] https://issues.jboss.org/browse/KEYCLOAK-7997 On Thu, Aug 2, 2018 at 2:45 PM Sebastien Blanc wrote: > So the first part , Client x509 Authentication is available here for review > https://github.com/keycloak/keycloak/pull/5435 , the client_id will be > lookup on the form data and the query parameters. > > And as Marek said I would like to follow up with another ticket for support > of OIDC Dynamic Client Registration and the support of the > `tls_client_auth_subject_dn`. > > Seb > > > On Wed, Aug 1, 2018 at 2:43 PM, Marek Posolda wrote: > > > On 30/07/18 09:20, Sebastien Blanc wrote: > > > >> Thanks Takashi, this is really useful information. Some comments inline > >> > >> On Mon, Jul 30, 2018 at 2:40 AM, ???? / NORIMATSU?TAKASHI < > >> takashi.norimatsu.ws at hitachi.com> wrote: > >> > >> Hello Sebastien, > >>> > >>> I've read the ticket you mentioned: https://issues.jboss.org/ > >>> browse/KEYCLOAK-7635 > >>> > >>> I'm not sure what the token review endpoint is (similar to RFC 7662 > OAuth > >>> 2.0 Token Introspection?), but I inferred the following situation. > >>> * OpenShift accesses keycloak's token review endpoint with some token, > >>> but > >>> without client_id query parameter or client_id encoded Authorization > >>> header. Therefore, keycloak cannot conduct client authentication. In > this > >>> situation, a client certificate might be used for client authentication > >>> in > >>> the context of OAuth 2.0. > >>> > >>> Also, I'm not sure how to enclose client_id or client_id identifiable > >>> information onto a client certificate and extract it. So I would like > to > >>> comment on other aspects other that it. > >>> > >>> - register client dn > >>> According to 2.1.2. Client Registration Metadata of OAuth 2.0 Mutual > TLS > >>> Client Authentication and Certificate Bound Access Tokens ( > >>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1.2), > >>> each > >>> client has "tls_client_auth_subject_dn" metadata parameter's value. > This > >>> has already been mentioned by @mposolda in https://issues.jboss.org/ > >>> browse/KEYCLOAK-7512 > >>> > >>> In keycloak, OIDCClientRepresentation defines these OIDC Dynamic client > >>> registration properties. > >>> > >>> OIDCAdvancedConfigWrapper can be used to convert these properties onto > >>> attributes of the client model. > >>> > >>> DescriptionConverter can be used to convert internal representation of > >>> the > >>> client onto external one (through OIDC Dynamic client registration) and > >>> vice versa. > >>> > >>> IMO, at least, those three classes have to be modified to support this > >>> new > >>> OIDC Client Registration property "tls_client_auth_subject_dn". > >>> > >>> This is a nice complement to what Marek said in the ticket and makes > >> thing > >> clearer for me as well. > >> But this would mean that Openshift should use dynamic client > registration > >> to make this flow works ? > >> > > > > Now I remember that few months ago, Bill implemented the support for > > ClientStorage SPI. And I think he implemented it mainly because of > > Openshift. It somehow allows to provision clients automatically from 3rd > > party source (EG. from Openshift) without a need to manually > > register/create them. So I guess (not 100% sure) that Openshift > integration > > will use this for client provisioning and it won't use dynamic client > > registration. > > > > IMO using dynamic client registration will be more aligned with the OIDC > > specs, but I guess we don't have a "power" to change the behaviour of > > Openshift. Or do we? :-) > > > > But support for the OIDC Dynamic client registration, which Takashi > > mentions, and which is mentioned in the specification, will be good to > have > > IMO even if it's not something directly useful for Openshift. If it's not > > priority for you and Openshift integration, we can maybe at least create > > some follow-up JIRA to add it later? > > > > Marek > > > >> > >> - extract client certificate > >>> > >>> X509ClientCertificateLookup : > >>> https://github.com/keycloak/keycloak/blob/master/services/ > >>> src/main/java/org/keycloak/services/x509/X509ClientCertifica > >>> teLookup.java > >>> This Client Certificate Lookup provider can be used to leverage the > >>> feature supporting reverse proxy deployed environment mentioned in the > >>> server admin manual ( > https://www.keycloak.org/docs/latest/server_admin/ > >>> index.html#client-certificate-lookup) . > >>> > >>> You can consult > >>> https://github.com/keycloak/keycloak/blob/master/services/ > >>> src/main/java/org/keycloak/authentication/authenticators/x509/ > >>> AbstractX509ClientCertificateAuthenticator.java#L196 > >>> to get how to use this client certificate lookup feature. > >>> > >>> In order to use it, KeycloakSession and HttpRequest instances are > >>> required > >>> to find the Client Certificate Lookup provider implementation. > >>> > >>> I've found that current ClientAuthenticator can get it by > >>> ClientAuthenticationFlowContext.getSession() and .getHttpRequest() in > >>> ClientAuthenticator.authenticateClient(ClientAuthenticationFlowContext > >>> context). > >>> > >>> Yes, I started to test that, exactly with this classes and methods but > >> until now I fail to retrieve any certificate (probably because of my > local > >> setup/configuration). > >> > >> - server side metadata advertisement > >>> > >>> OIDCConfigurationRepresentation is a server side metadata > representation > >>> and might be related to 2.1.1. PKI Authentication Method Metadata Value > >>> of > >>> OAuth 2.0 Mutual TLS Client Authentication and Certificate Bound Access > >>> Tokens (https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#sectio > >>> n-2.1.1 > >>> ) > >>> > >>> Best regards, > >>> Takashi Norimatsu > >>> Hitachi Ltd., > >>> > >>> ---------- > >>> From: Sebastien Blanc > >>> Sent: Friday, July 27, 2018 4:31 PM > >>> To: ???? / NORIMATSU?TAKASHI > >>> Cc: Sebastian Laskawiec ; > >>> keycloak-dev at lists.jboss.org > >>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client > Authentication > >>> > >>> Hi Takashi ! > >>> > >>> You can even help before if you want to :) > >>> > >>> The ticket is here : https://issues.jboss.org/browse/KEYCLOAK-7635 > >>> > >>> I created an "empty" X509ClientAuthenticator" branch here : > >>> https://github.com/sebastienblanc/keycloak/tree/client-x509 > >>> > >>> And I'm investigating what we can reuse from this pacakge : > >>> https://github.com/keycloak/keycloak/tree/master/services/ > >>> src/main/java/org/keycloak/authentication/authenticators/x509 > >>> > >>> Any feedback, help, advise is welcome ! > >>> > >>> Sebi > >>> > >>> > >>> On Fri, Jul 27, 2018 at 3:22 AM, ???? / NORIMATSU?TAKASHI < > >>> takashi.norimatsu.ws at hitachi.com> wrote: > >>> > >>> Hello Sebastian, > >>>> > >>>> I'm looking forward to your work, and I would be happy if I could make > >>>> some contribution after finishing your work. > >>>> > >>>> Best regards, > >>>> Takashi Norimatsu > >>>> Hitachi Ltd., > >>>> > >>>> ---------- > >>>> From: Sebastian Laskawiec > >>>> Sent: Thursday, July 26, 2018 5:24 PM > >>>> To: ???? / NORIMATSU?TAKASHI > >>>> Cc: keycloak-dev at lists.jboss.org > >>>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client > >>>> Authentication > >>>> > >>>> Hey Takashi, > >>>> > >>>> Thanks a lot for the interest in contributing Keycloak! > >>>> > >>>> Sebi and I are working on this topic currently. We plan to reuse some > >>>> > >>> bits > >>> > >>>> of the User x509 Authentication and bring them to the client. We > planned > >>>> the implementation for this sprint, so it *should* be ready in ~3 > weeks. > >>>> > >>>> More comments inlined. > >>>> > >>>> Thanks, > >>>> Sebastian > >>>> On Thu, Jul 26, 2018 at 1:23 AM ???? / NORIMATSU?TAKASHI < > >>>> takashi.norimatsu.ws at hitachi.com> wrote: > >>>> Hello, > >>>> > >>>> As for mentioned in https://issues.jboss.org/browse/KEYCLOAK-7512 and > >>>> https://issues.jboss.org/browse/KEYCLOAK-7635, Is there anyone who > >>>> currently implements OAuth 2.0 Mutual TLS Client Authentication > defined > >>>> > >>> in > >>> > >>>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 ? > >>>> > >>>> We also have additional requirement - allow to authenticate client > >>>> > >>> without > >>> > >>>> "client_id" being sent (we need to extract it from the Certificate > >>>> > >>> obtained > >>> > >>>> during TLS Handshake). This is required for OpenShift integration. > >>>> > >>>> > >>>> If no one does it, I would like to try to implement this feature. What > >>>> do > >>>> you think about it ? > >>>> > >>>> Also, In > https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 > >>>> , > >>>> two types of OAuth 2.0 Mutual TLS Client Authentication are defined, > for > >>>> PKI and for Self-Signed Certificate. > >>>> > >>>> I would be happy if you who are interested in this feature tell me > which > >>>> you like better. > >>>> > >>>> As far as I know, we won't be touching self-registering clients. So > >>>> maybe > >>>> once we are done (let's assume that will happen in ~3 weeks), you > could > >>>> take it over and look into that? > >>>> > >>>> BTW, as for now, we will be implementing everything in this branch: > >>>> https://github.com/sebastienblanc/keycloak/tree/client-x509 > (currently, > >>>> it contains an empty Authenticator but we will be adding bits and > pieces > >>>> > >>> to > >>> > >>>> it). > >>>> > >>>> > >>>> Best regards, > >>>> Takashi Norimatsu > >>>> Hitachi Ltd., > >>>> > >>>> _______________________________________________ > >>>> keycloak-dev mailing list > >>>> keycloak-dev at lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>>> > >>>> _______________________________________________ > >>>> keycloak-dev mailing list > >>>> keycloak-dev at lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>>> > >>>> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From dt at acutus.pro Mon Aug 6 09:34:20 2018 From: dt at acutus.pro (Dmitry Telegin) Date: Mon, 06 Aug 2018 16:34:20 +0300 Subject: [keycloak-dev] Authentication flow deep/shallow copy ambiguity Message-ID: <1533562460.13393.8.camel@acutus.pro> Hi, When copying authentication flow where some authenticators have been configured, I'd expect deep copy, i.e. including authenticator configs. But that's not the case, here are the steps to reproduce: - create Copy of Browser Flow; - configure Identity Provider Redirector with "foo" IdP; - create Copy of Copy of Browser flow; - go to Identity Provider Redirector config, change "foo" to "bar"; - go back to Copy of Browser Flow. The redirector config will change to "bar". Is this a bug? If not, I think this should be documented, because it's not obvious and can cause confusion. Thanks, Dmitry From mposolda at redhat.com Tue Aug 7 02:10:28 2018 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 7 Aug 2018 08:10:28 +0200 Subject: [keycloak-dev] Authentication flow deep/shallow copy ambiguity In-Reply-To: <1533562460.13393.8.camel@acutus.pro> References: <1533562460.13393.8.camel@acutus.pro> Message-ID: <92ae2968-f8bd-0e67-db38-3b2f7f84d600@redhat.com> Hi Dmitry, yes this looks like a bug to me. Feel free to create JIRA. Even better PR with the test ;) (There are some tests in the org.keycloak.testsuite.admin.authentication package and I hope the test for this could be added here. Marek On 06/08/18 15:34, Dmitry Telegin wrote: > Hi, > > When copying authentication flow where some authenticators have been > configured, I'd expect deep copy, i.e. including authenticator configs. > But that's not the case, here are the steps to reproduce: > > - create Copy of Browser Flow; > - configure Identity Provider Redirector with "foo" IdP; > - create Copy of Copy of Browser flow; > - go to Identity Provider Redirector config, change "foo" to "bar"; > - go back to Copy of Browser Flow. The redirector config will change to > "bar". > > Is this a bug? If not, I think this should be documented, because it's > not obvious and can cause confusion. > > Thanks, > Dmitry > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From takashi.norimatsu.ws at hitachi.com Tue Aug 7 03:04:46 2018 From: takashi.norimatsu.ws at hitachi.com (=?utf-8?B?5LmX5p2+6ZqG5b+XIC8gTk9SSU1BVFNV77yMVEFLQVNISQ==?=) Date: Tue, 7 Aug 2018 07:04:46 +0000 Subject: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication In-Reply-To: References: Message-ID: Hello Sebastian, I've read the follow-up JIRA (https://issues.jboss.org/browse/KEYCLOAK-7997). It might be better to implement PKI Mutual TLS OAuth Client Authentication Method defined in the specification in https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1 on your PR https://github.com/keycloak/keycloak/pull/5435 . Or, after your current PR is merged (namely, this PR is for OpenShift Integration), PKI Mutual TLS OAuth Client Authentication Method will be implemented over it. Best regards, Takashi Norimatsu Hitachi Ltd., -----Original Message----- From: keycloak-dev-bounces at lists.jboss.org On Behalf Of Sebastian Laskawiec Sent: Monday, August 06, 2018 6:12 PM To: Sebastien Blanc Cc: keycloak-dev at lists.jboss.org Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication Hey Takashi, As the initial version is more or less done, I created a follow-up JIRA with dynamic client registration and Mutual TLS [1]. Perhaps you'd be interested in looking into it? Thanks, Sebastian [1] https://issues.jboss.org/browse/KEYCLOAK-7997 On Thu, Aug 2, 2018 at 2:45 PM Sebastien Blanc wrote: > So the first part , Client x509 Authentication is available here for review > https://github.com/keycloak/keycloak/pull/5435 , the client_id will be > lookup on the form data and the query parameters. > > And as Marek said I would like to follow up with another ticket for support > of OIDC Dynamic Client Registration and the support of the > `tls_client_auth_subject_dn`. > > Seb > > > On Wed, Aug 1, 2018 at 2:43 PM, Marek Posolda wrote: > > > On 30/07/18 09:20, Sebastien Blanc wrote: > > > >> Thanks Takashi, this is really useful information. Some comments inline > >> > >> On Mon, Jul 30, 2018 at 2:40 AM, ???? / NORIMATSU?TAKASHI < > >> takashi.norimatsu.ws at hitachi.com> wrote: > >> > >> Hello Sebastien, > >>> > >>> I've read the ticket you mentioned: https://issues.jboss.org/ > >>> browse/KEYCLOAK-7635 > >>> > >>> I'm not sure what the token review endpoint is (similar to RFC 7662 > OAuth > >>> 2.0 Token Introspection?), but I inferred the following situation. > >>> * OpenShift accesses keycloak's token review endpoint with some token, > >>> but > >>> without client_id query parameter or client_id encoded Authorization > >>> header. Therefore, keycloak cannot conduct client authentication. In > this > >>> situation, a client certificate might be used for client authentication > >>> in > >>> the context of OAuth 2.0. > >>> > >>> Also, I'm not sure how to enclose client_id or client_id identifiable > >>> information onto a client certificate and extract it. So I would like > to > >>> comment on other aspects other that it. > >>> > >>> - register client dn > >>> According to 2.1.2. Client Registration Metadata of OAuth 2.0 Mutual > TLS > >>> Client Authentication and Certificate Bound Access Tokens ( > >>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1.2), > >>> each > >>> client has "tls_client_auth_subject_dn" metadata parameter's value. > This > >>> has already been mentioned by @mposolda in https://issues.jboss.org/ > >>> browse/KEYCLOAK-7512 > >>> > >>> In keycloak, OIDCClientRepresentation defines these OIDC Dynamic client > >>> registration properties. > >>> > >>> OIDCAdvancedConfigWrapper can be used to convert these properties onto > >>> attributes of the client model. > >>> > >>> DescriptionConverter can be used to convert internal representation of > >>> the > >>> client onto external one (through OIDC Dynamic client registration) and > >>> vice versa. > >>> > >>> IMO, at least, those three classes have to be modified to support this > >>> new > >>> OIDC Client Registration property "tls_client_auth_subject_dn". > >>> > >>> This is a nice complement to what Marek said in the ticket and makes > >> thing > >> clearer for me as well. > >> But this would mean that Openshift should use dynamic client > registration > >> to make this flow works ? > >> > > > > Now I remember that few months ago, Bill implemented the support for > > ClientStorage SPI. And I think he implemented it mainly because of > > Openshift. It somehow allows to provision clients automatically from 3rd > > party source (EG. from Openshift) without a need to manually > > register/create them. So I guess (not 100% sure) that Openshift > integration > > will use this for client provisioning and it won't use dynamic client > > registration. > > > > IMO using dynamic client registration will be more aligned with the OIDC > > specs, but I guess we don't have a "power" to change the behaviour of > > Openshift. Or do we? :-) > > > > But support for the OIDC Dynamic client registration, which Takashi > > mentions, and which is mentioned in the specification, will be good to > have > > IMO even if it's not something directly useful for Openshift. If it's not > > priority for you and Openshift integration, we can maybe at least create > > some follow-up JIRA to add it later? > > > > Marek > > > >> > >> - extract client certificate > >>> > >>> X509ClientCertificateLookup : > >>> https://github.com/keycloak/keycloak/blob/master/services/ > >>> src/main/java/org/keycloak/services/x509/X509ClientCertifica > >>> teLookup.java > >>> This Client Certificate Lookup provider can be used to leverage the > >>> feature supporting reverse proxy deployed environment mentioned in the > >>> server admin manual ( > https://www.keycloak.org/docs/latest/server_admin/ > >>> index.html#client-certificate-lookup) . > >>> > >>> You can consult > >>> https://github.com/keycloak/keycloak/blob/master/services/ > >>> src/main/java/org/keycloak/authentication/authenticators/x509/ > >>> AbstractX509ClientCertificateAuthenticator.java#L196 > >>> to get how to use this client certificate lookup feature. > >>> > >>> In order to use it, KeycloakSession and HttpRequest instances are > >>> required > >>> to find the Client Certificate Lookup provider implementation. > >>> > >>> I've found that current ClientAuthenticator can get it by > >>> ClientAuthenticationFlowContext.getSession() and .getHttpRequest() in > >>> ClientAuthenticator.authenticateClient(ClientAuthenticationFlowContext > >>> context). > >>> > >>> Yes, I started to test that, exactly with this classes and methods but > >> until now I fail to retrieve any certificate (probably because of my > local > >> setup/configuration). > >> > >> - server side metadata advertisement > >>> > >>> OIDCConfigurationRepresentation is a server side metadata > representation > >>> and might be related to 2.1.1. PKI Authentication Method Metadata Value > >>> of > >>> OAuth 2.0 Mutual TLS Client Authentication and Certificate Bound Access > >>> Tokens (https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#sectio > >>> n-2.1.1 > >>> ) > >>> > >>> Best regards, > >>> Takashi Norimatsu > >>> Hitachi Ltd., > >>> > >>> ---------- > >>> From: Sebastien Blanc > >>> Sent: Friday, July 27, 2018 4:31 PM > >>> To: ???? / NORIMATSU?TAKASHI > >>> Cc: Sebastian Laskawiec ; > >>> keycloak-dev at lists.jboss.org > >>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client > Authentication > >>> > >>> Hi Takashi ! > >>> > >>> You can even help before if you want to :) > >>> > >>> The ticket is here : https://issues.jboss.org/browse/KEYCLOAK-7635 > >>> > >>> I created an "empty" X509ClientAuthenticator" branch here : > >>> https://github.com/sebastienblanc/keycloak/tree/client-x509 > >>> > >>> And I'm investigating what we can reuse from this pacakge : > >>> https://github.com/keycloak/keycloak/tree/master/services/ > >>> src/main/java/org/keycloak/authentication/authenticators/x509 > >>> > >>> Any feedback, help, advise is welcome ! > >>> > >>> Sebi > >>> > >>> > >>> On Fri, Jul 27, 2018 at 3:22 AM, ???? / NORIMATSU?TAKASHI < > >>> takashi.norimatsu.ws at hitachi.com> wrote: > >>> > >>> Hello Sebastian, > >>>> > >>>> I'm looking forward to your work, and I would be happy if I could make > >>>> some contribution after finishing your work. > >>>> > >>>> Best regards, > >>>> Takashi Norimatsu > >>>> Hitachi Ltd., > >>>> > >>>> ---------- > >>>> From: Sebastian Laskawiec > >>>> Sent: Thursday, July 26, 2018 5:24 PM > >>>> To: ???? / NORIMATSU?TAKASHI > >>>> Cc: keycloak-dev at lists.jboss.org > >>>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client > >>>> Authentication > >>>> > >>>> Hey Takashi, > >>>> > >>>> Thanks a lot for the interest in contributing Keycloak! > >>>> > >>>> Sebi and I are working on this topic currently. We plan to reuse some > >>>> > >>> bits > >>> > >>>> of the User x509 Authentication and bring them to the client. We > planned > >>>> the implementation for this sprint, so it *should* be ready in ~3 > weeks. > >>>> > >>>> More comments inlined. > >>>> > >>>> Thanks, > >>>> Sebastian > >>>> On Thu, Jul 26, 2018 at 1:23 AM ???? / NORIMATSU?TAKASHI < > >>>> takashi.norimatsu.ws at hitachi.com> wrote: > >>>> Hello, > >>>> > >>>> As for mentioned in https://issues.jboss.org/browse/KEYCLOAK-7512 and > >>>> https://issues.jboss.org/browse/KEYCLOAK-7635, Is there anyone who > >>>> currently implements OAuth 2.0 Mutual TLS Client Authentication > defined > >>>> > >>> in > >>> > >>>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 ? > >>>> > >>>> We also have additional requirement - allow to authenticate client > >>>> > >>> without > >>> > >>>> "client_id" being sent (we need to extract it from the Certificate > >>>> > >>> obtained > >>> > >>>> during TLS Handshake). This is required for OpenShift integration. > >>>> > >>>> > >>>> If no one does it, I would like to try to implement this feature. What > >>>> do > >>>> you think about it ? > >>>> > >>>> Also, In > https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 > >>>> , > >>>> two types of OAuth 2.0 Mutual TLS Client Authentication are defined, > for > >>>> PKI and for Self-Signed Certificate. > >>>> > >>>> I would be happy if you who are interested in this feature tell me > which > >>>> you like better. > >>>> > >>>> As far as I know, we won't be touching self-registering clients. So > >>>> maybe > >>>> once we are done (let's assume that will happen in ~3 weeks), you > could > >>>> take it over and look into that? > >>>> > >>>> BTW, as for now, we will be implementing everything in this branch: > >>>> https://github.com/sebastienblanc/keycloak/tree/client-x509 > (currently, > >>>> it contains an empty Authenticator but we will be adding bits and > pieces > >>>> > >>> to > >>> > >>>> it). > >>>> > >>>> > >>>> Best regards, > >>>> Takashi Norimatsu > >>>> Hitachi Ltd., > >>>> > >>>> _______________________________________________ > >>>> keycloak-dev mailing list > >>>> keycloak-dev at lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>>> > >>>> _______________________________________________ > >>>> keycloak-dev mailing list > >>>> keycloak-dev at lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>>> > >>>> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From slaskawi at redhat.com Tue Aug 7 03:09:10 2018 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Tue, 7 Aug 2018 09:09:10 +0200 Subject: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication In-Reply-To: References: Message-ID: On Tue, Aug 7, 2018 at 9:05 AM ???? / NORIMATSU?TAKASHI < takashi.norimatsu.ws at hitachi.com> wrote: > Hello Sebastian, > > I've read the follow-up JIRA ( > https://issues.jboss.org/browse/KEYCLOAK-7997). > > It might be better to implement PKI Mutual TLS OAuth Client Authentication > Method defined in the specification in > https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1 on your > PR https://github.com/keycloak/keycloak/pull/5435 . > > Or, after your current PR is merged (namely, this PR is for OpenShift > Integration), PKI Mutual TLS OAuth Client Authentication Method will be > implemented over it. > Yes, I would advocate for this approach. We need the first part for OpenShift Integration whereas the second part (the dynamic client) is optional in this context. > > Best regards, > Takashi Norimatsu > Hitachi Ltd., > > -----Original Message----- > From: keycloak-dev-bounces at lists.jboss.org < > keycloak-dev-bounces at lists.jboss.org> On Behalf Of Sebastian Laskawiec > Sent: Monday, August 06, 2018 6:12 PM > To: Sebastien Blanc > Cc: keycloak-dev at lists.jboss.org > Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client Authentication > > Hey Takashi, > > As the initial version is more or less done, I created a follow-up JIRA > with dynamic client registration and Mutual TLS [1]. Perhaps you'd be > interested in looking into it? > > Thanks, > Sebastian > > [1] https://issues.jboss.org/browse/KEYCLOAK-7997 > > On Thu, Aug 2, 2018 at 2:45 PM Sebastien Blanc > wrote: > > > So the first part , Client x509 Authentication is available here for > review > > https://github.com/keycloak/keycloak/pull/5435 , the client_id will be > > lookup on the form data and the query parameters. > > > > And as Marek said I would like to follow up with another ticket for > support > > of OIDC Dynamic Client Registration and the support of the > > `tls_client_auth_subject_dn`. > > > > Seb > > > > > > On Wed, Aug 1, 2018 at 2:43 PM, Marek Posolda > wrote: > > > > > On 30/07/18 09:20, Sebastien Blanc wrote: > > > > > >> Thanks Takashi, this is really useful information. Some comments > inline > > >> > > >> On Mon, Jul 30, 2018 at 2:40 AM, ???? / NORIMATSU?TAKASHI < > > >> takashi.norimatsu.ws at hitachi.com> wrote: > > >> > > >> Hello Sebastien, > > >>> > > >>> I've read the ticket you mentioned: https://issues.jboss.org/ > > >>> browse/KEYCLOAK-7635 > > >>> > > >>> I'm not sure what the token review endpoint is (similar to RFC 7662 > > OAuth > > >>> 2.0 Token Introspection?), but I inferred the following situation. > > >>> * OpenShift accesses keycloak's token review endpoint with some > token, > > >>> but > > >>> without client_id query parameter or client_id encoded Authorization > > >>> header. Therefore, keycloak cannot conduct client authentication. In > > this > > >>> situation, a client certificate might be used for client > authentication > > >>> in > > >>> the context of OAuth 2.0. > > >>> > > >>> Also, I'm not sure how to enclose client_id or client_id identifiable > > >>> information onto a client certificate and extract it. So I would like > > to > > >>> comment on other aspects other that it. > > >>> > > >>> - register client dn > > >>> According to 2.1.2. Client Registration Metadata of OAuth 2.0 Mutual > > TLS > > >>> Client Authentication and Certificate Bound Access Tokens ( > > >>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#section-2.1.2), > > >>> each > > >>> client has "tls_client_auth_subject_dn" metadata parameter's value. > > This > > >>> has already been mentioned by @mposolda in https://issues.jboss.org/ > > >>> browse/KEYCLOAK-7512 > > >>> > > >>> In keycloak, OIDCClientRepresentation defines these OIDC Dynamic > client > > >>> registration properties. > > >>> > > >>> OIDCAdvancedConfigWrapper can be used to convert these properties > onto > > >>> attributes of the client model. > > >>> > > >>> DescriptionConverter can be used to convert internal representation > of > > >>> the > > >>> client onto external one (through OIDC Dynamic client registration) > and > > >>> vice versa. > > >>> > > >>> IMO, at least, those three classes have to be modified to support > this > > >>> new > > >>> OIDC Client Registration property "tls_client_auth_subject_dn". > > >>> > > >>> This is a nice complement to what Marek said in the ticket and makes > > >> thing > > >> clearer for me as well. > > >> But this would mean that Openshift should use dynamic client > > registration > > >> to make this flow works ? > > >> > > > > > > Now I remember that few months ago, Bill implemented the support for > > > ClientStorage SPI. And I think he implemented it mainly because of > > > Openshift. It somehow allows to provision clients automatically from > 3rd > > > party source (EG. from Openshift) without a need to manually > > > register/create them. So I guess (not 100% sure) that Openshift > > integration > > > will use this for client provisioning and it won't use dynamic client > > > registration. > > > > > > IMO using dynamic client registration will be more aligned with the > OIDC > > > specs, but I guess we don't have a "power" to change the behaviour of > > > Openshift. Or do we? :-) > > > > > > But support for the OIDC Dynamic client registration, which Takashi > > > mentions, and which is mentioned in the specification, will be good to > > have > > > IMO even if it's not something directly useful for Openshift. If it's > not > > > priority for you and Openshift integration, we can maybe at least > create > > > some follow-up JIRA to add it later? > > > > > > Marek > > > > > >> > > >> - extract client certificate > > >>> > > >>> X509ClientCertificateLookup : > > >>> https://github.com/keycloak/keycloak/blob/master/services/ > > >>> src/main/java/org/keycloak/services/x509/X509ClientCertifica > > >>> teLookup.java > > >>> This Client Certificate Lookup provider can be used to leverage the > > >>> feature supporting reverse proxy deployed environment mentioned in > the > > >>> server admin manual ( > > https://www.keycloak.org/docs/latest/server_admin/ > > >>> index.html#client-certificate-lookup) . > > >>> > > >>> You can consult > > >>> https://github.com/keycloak/keycloak/blob/master/services/ > > >>> src/main/java/org/keycloak/authentication/authenticators/x509/ > > >>> AbstractX509ClientCertificateAuthenticator.java#L196 > > >>> to get how to use this client certificate lookup feature. > > >>> > > >>> In order to use it, KeycloakSession and HttpRequest instances are > > >>> required > > >>> to find the Client Certificate Lookup provider implementation. > > >>> > > >>> I've found that current ClientAuthenticator can get it by > > >>> ClientAuthenticationFlowContext.getSession() and .getHttpRequest() in > > >>> > ClientAuthenticator.authenticateClient(ClientAuthenticationFlowContext > > >>> context). > > >>> > > >>> Yes, I started to test that, exactly with this classes and methods > but > > >> until now I fail to retrieve any certificate (probably because of my > > local > > >> setup/configuration). > > >> > > >> - server side metadata advertisement > > >>> > > >>> OIDCConfigurationRepresentation is a server side metadata > > representation > > >>> and might be related to 2.1.1. PKI Authentication Method Metadata > Value > > >>> of > > >>> OAuth 2.0 Mutual TLS Client Authentication and Certificate Bound > Access > > >>> Tokens (https://tools.ietf.org/html/draft-ietf-oauth-mtls-10#sectio > > >>> n-2.1.1 > > >>> ) > > >>> > > >>> Best regards, > > >>> Takashi Norimatsu > > >>> Hitachi Ltd., > > >>> > > >>> ---------- > > >>> From: Sebastien Blanc > > >>> Sent: Friday, July 27, 2018 4:31 PM > > >>> To: ???? / NORIMATSU?TAKASHI > > >>> Cc: Sebastian Laskawiec ; > > >>> keycloak-dev at lists.jboss.org > > >>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client > > Authentication > > >>> > > >>> Hi Takashi ! > > >>> > > >>> You can even help before if you want to :) > > >>> > > >>> The ticket is here : https://issues.jboss.org/browse/KEYCLOAK-7635 > > >>> > > >>> I created an "empty" X509ClientAuthenticator" branch here : > > >>> https://github.com/sebastienblanc/keycloak/tree/client-x509 > > >>> > > >>> And I'm investigating what we can reuse from this pacakge : > > >>> https://github.com/keycloak/keycloak/tree/master/services/ > > >>> src/main/java/org/keycloak/authentication/authenticators/x509 > > >>> > > >>> Any feedback, help, advise is welcome ! > > >>> > > >>> Sebi > > >>> > > >>> > > >>> On Fri, Jul 27, 2018 at 3:22 AM, ???? / NORIMATSU?TAKASHI < > > >>> takashi.norimatsu.ws at hitachi.com> wrote: > > >>> > > >>> Hello Sebastian, > > >>>> > > >>>> I'm looking forward to your work, and I would be happy if I could > make > > >>>> some contribution after finishing your work. > > >>>> > > >>>> Best regards, > > >>>> Takashi Norimatsu > > >>>> Hitachi Ltd., > > >>>> > > >>>> ---------- > > >>>> From: Sebastian Laskawiec > > >>>> Sent: Thursday, July 26, 2018 5:24 PM > > >>>> To: ???? / NORIMATSU?TAKASHI > > >>>> Cc: keycloak-dev at lists.jboss.org > > >>>> Subject: [!]Re: [keycloak-dev] OAuth 2.0 Mutual TLS Client > > >>>> Authentication > > >>>> > > >>>> Hey Takashi, > > >>>> > > >>>> Thanks a lot for the interest in contributing Keycloak! > > >>>> > > >>>> Sebi and I are working on this topic currently. We plan to reuse > some > > >>>> > > >>> bits > > >>> > > >>>> of the User x509 Authentication and bring them to the client. We > > planned > > >>>> the implementation for this sprint, so it *should* be ready in ~3 > > weeks. > > >>>> > > >>>> More comments inlined. > > >>>> > > >>>> Thanks, > > >>>> Sebastian > > >>>> On Thu, Jul 26, 2018 at 1:23 AM ???? / NORIMATSU?TAKASHI < > > >>>> takashi.norimatsu.ws at hitachi.com> wrote: > > >>>> Hello, > > >>>> > > >>>> As for mentioned in https://issues.jboss.org/browse/KEYCLOAK-7512 > and > > >>>> https://issues.jboss.org/browse/KEYCLOAK-7635, Is there anyone who > > >>>> currently implements OAuth 2.0 Mutual TLS Client Authentication > > defined > > >>>> > > >>> in > > >>> > > >>>> https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 ? > > >>>> > > >>>> We also have additional requirement - allow to authenticate client > > >>>> > > >>> without > > >>> > > >>>> "client_id" being sent (we need to extract it from the Certificate > > >>>> > > >>> obtained > > >>> > > >>>> during TLS Handshake). This is required for OpenShift integration. > > >>>> > > >>>> > > >>>> If no one does it, I would like to try to implement this feature. > What > > >>>> do > > >>>> you think about it ? > > >>>> > > >>>> Also, In > > https://tools.ietf.org/html/draft-ietf-oauth-mtls-07#section-2 > > >>>> , > > >>>> two types of OAuth 2.0 Mutual TLS Client Authentication are defined, > > for > > >>>> PKI and for Self-Signed Certificate. > > >>>> > > >>>> I would be happy if you who are interested in this feature tell me > > which > > >>>> you like better. > > >>>> > > >>>> As far as I know, we won't be touching self-registering clients. So > > >>>> maybe > > >>>> once we are done (let's assume that will happen in ~3 weeks), you > > could > > >>>> take it over and look into that? > > >>>> > > >>>> BTW, as for now, we will be implementing everything in this branch: > > >>>> https://github.com/sebastienblanc/keycloak/tree/client-x509 > > (currently, > > >>>> it contains an empty Authenticator but we will be adding bits and > > pieces > > >>>> > > >>> to > > >>> > > >>>> it). > > >>>> > > >>>> > > >>>> Best regards, > > >>>> Takashi Norimatsu > > >>>> Hitachi Ltd., > > >>>> > > >>>> _______________________________________________ > > >>>> keycloak-dev mailing list > > >>>> keycloak-dev at lists.jboss.org > > >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > >>>> > > >>>> _______________________________________________ > > >>>> keycloak-dev mailing list > > >>>> keycloak-dev at lists.jboss.org > > >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > >>>> > > >>>> _______________________________________________ > > >> keycloak-dev mailing list > > >> keycloak-dev at lists.jboss.org > > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > >> > > > > > > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From Sebastian.Schuster at bosch-si.com Tue Aug 7 11:08:10 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Tue, 7 Aug 2018 15:08:10 +0000 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions Message-ID: Hi everybody, I just noticed that 4.2.1 contains a migration (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column from the RESOURCE_SERVER_RESOURCE table and puts it into a separate table RESOURCE_URIS. This table has a NOT NULL constraint on the new uri column (called VALUE). The accompanying data migration AuthzResourceUseMoreURIs.java selects rows from the old table and inserts URIs it into the new. This fails for all resources that did not have a URI before because of the NOT NULL constraint, for example for Keycloak-internal resources like groups that don?t have a URI. Is this intended behavior? Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn From thomas.darimont at googlemail.com Tue Aug 7 12:09:09 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 7 Aug 2018 18:09:09 +0200 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: References: Message-ID: Hello, I was just bitten by this as well 3hours ago, but thankfully only in our staging environment. We had only one entry in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri and icon_uri column. This caused the migration to fail. In our prod env I there was no entry in that table, so the migration went through. As a quick fix in the staging env I just changed those uris to http://doesnotexist.local and http://doesnotexist.local/icon respectively to see make it pass. It seems that I triggered the creation of those entries in the RESOURCE_SERVER_RESOURCE table when I activated and deactivated the authz support for a client. I think this should be addressed in the migrations. There should be at least a note about that in the migration guides. It took me a while to find the table that contained the null values that were indirectly causing the migration to fail. Cheers, Thomas On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < Sebastian.Schuster at bosch-si.com> wrote: > Hi everybody, > > I just noticed that 4.2.1 contains a migration > (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column from the > RESOURCE_SERVER_RESOURCE table and puts it into a separate table > RESOURCE_URIS. This table has a NOT NULL constraint on the new uri column > (called VALUE). The accompanying data migration > AuthzResourceUseMoreURIs.java selects rows from the old table and inserts > URIs it into the new. This fails for all resources that did not have a URI > before because of the NOT NULL constraint, for example for > Keycloak-internal resources like groups that don?t have a URI. > > Is this intended behavior? > > Best regards, > Sebastian > > Mit freundlichen Gr??en / Best regards > > Dr.-Ing. Sebastian Schuster > > Engineering and Support (INST/ESY1) > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > GERMANY | www.bosch-si.com > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > Sebastian.Schuster at bosch-si.com > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > Stefan Ferber, Michael Hahn > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From francoismaturel at dijit.fr Tue Aug 7 12:44:08 2018 From: francoismaturel at dijit.fr (francois maturel) Date: Tue, 7 Aug 2018 18:44:08 +0200 Subject: [keycloak-dev] Need some guidance on authenticating from third party website Message-ID: <63684805-c1f9-98cc-ea9c-499d00262872@dijit.fr> Hello team! We have a solution based on Keycloak 3.4.3.Final where our customer used to login through to the standard Keycloak login form. This customer now wants to pre-authenticate their users to our solution when they log into their own portal. The customer provides us all their user's username / password for our solution (that we import through a batch) and they can use those to authenticate theirs users in our solution. Note that our customer have different username / password for their portal. We proposed to adapt the login form Content-Security-Policy to allow the login form to load in their portal (with login_hint), but this still requires a user action. Is there any way to log into Keycloak automatically from a third party website and get the KEYCLOAK_SESSION / IDENTITY cookies (we would like to avoid Direct Grant / Implicit Flow...) ? Could you please provide some guidance on how to achieve this? Thanks! -- Fran?ois Maturel Regards, Fran?ois Maturel From psilva at redhat.com Tue Aug 7 14:06:03 2018 From: psilva at redhat.com (Pedro Igor Silva) Date: Tue, 7 Aug 2018 15:06:03 -0300 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: References: Message-ID: Will fix that. Also looking why migration test is not catching this ... On Tue, Aug 7, 2018 at 1:09 PM, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hello, > > I was just bitten by this as well 3hours ago, but thankfully only in our > staging environment. We had only one entry > in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri and > icon_uri column. > This caused the migration to fail. In our prod env I there was no entry in > that table, so the migration went through. > As a quick fix in the staging env I just changed those uris to > http://doesnotexist.local and http://doesnotexist.local/icon respectively > to see make it pass. > > It seems that I triggered the creation of those entries in the > RESOURCE_SERVER_RESOURCE table when > I activated and deactivated the authz support for a client. > > I think this should be addressed in the migrations. There should be at > least a note about that in the migration guides. > It took me a while to find the table that contained the null values that > were indirectly causing the migration to fail. > > Cheers, > Thomas > > On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> wrote: > > > Hi everybody, > > > > I just noticed that 4.2.1 contains a migration > > (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column from > the > > RESOURCE_SERVER_RESOURCE table and puts it into a separate table > > RESOURCE_URIS. This table has a NOT NULL constraint on the new uri column > > (called VALUE). The accompanying data migration > > AuthzResourceUseMoreURIs.java selects rows from the old table and inserts > > URIs it into the new. This fails for all resources that did not have a > URI > > before because of the NOT NULL constraint, for example for > > Keycloak-internal resources like groups that don?t have a URI. > > > > Is this intended behavior? > > > > Best regards, > > Sebastian > > > > Mit freundlichen Gr??en / Best regards > > > > Dr.-Ing. Sebastian Schuster > > > > Engineering and Support (INST/ESY1) > > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > > GERMANY | www.bosch-si.com > > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > > Sebastian.Schuster at bosch-si.com > > > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > > Stefan Ferber, Michael Hahn > > > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From hmlnarik at redhat.com Tue Aug 7 14:08:08 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Tue, 7 Aug 2018 20:08:08 +0200 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: References: Message-ID: Apologies for this oversight, this will be fixed in the next version. https://issues.jboss.org/browse/KEYCLOAK-8003 On Tue, Aug 7, 2018 at 7:00 PM Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hello, > > I was just bitten by this as well 3hours ago, but thankfully only in our > staging environment. We had only one entry > in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri and > icon_uri column. > This caused the migration to fail. In our prod env I there was no entry in > that table, so the migration went through. > As a quick fix in the staging env I just changed those uris to > http://doesnotexist.local and http://doesnotexist.local/icon respectively > to see make it pass. > > It seems that I triggered the creation of those entries in the > RESOURCE_SERVER_RESOURCE table when > I activated and deactivated the authz support for a client. > > I think this should be addressed in the migrations. There should be at > least a note about that in the migration guides. > It took me a while to find the table that contained the null values that > were indirectly causing the migration to fail. > > Cheers, > Thomas > > On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> wrote: > > > Hi everybody, > > > > I just noticed that 4.2.1 contains a migration > > (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column from > the > > RESOURCE_SERVER_RESOURCE table and puts it into a separate table > > RESOURCE_URIS. This table has a NOT NULL constraint on the new uri column > > (called VALUE). The accompanying data migration > > AuthzResourceUseMoreURIs.java selects rows from the old table and inserts > > URIs it into the new. This fails for all resources that did not have a > URI > > before because of the NOT NULL constraint, for example for > > Keycloak-internal resources like groups that don?t have a URI. > > > > Is this intended behavior? > > > > Best regards, > > Sebastian > > > > Mit freundlichen Gr??en / Best regards > > > > Dr.-Ing. Sebastian Schuster > > > > Engineering and Support (INST/ESY1) > > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > > GERMANY | www.bosch-si.com > > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > > Sebastian.Schuster at bosch-si.com > > > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > > Stefan Ferber, Michael Hahn > > > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From psilva at redhat.com Tue Aug 7 18:31:02 2018 From: psilva at redhat.com (Pedro Igor Silva) Date: Tue, 7 Aug 2018 19:31:02 -0300 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: References: Message-ID: Sent a PR: https://github.com/keycloak/keycloak/pull/5446. On Tue, Aug 7, 2018 at 3:08 PM, Hynek Mlnarik wrote: > Apologies for this oversight, this will be fixed in the next version. > > https://issues.jboss.org/browse/KEYCLOAK-8003 > > On Tue, Aug 7, 2018 at 7:00 PM Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > > > Hello, > > > > I was just bitten by this as well 3hours ago, but thankfully only in our > > staging environment. We had only one entry > > in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri > and > > icon_uri column. > > This caused the migration to fail. In our prod env I there was no entry > in > > that table, so the migration went through. > > As a quick fix in the staging env I just changed those uris to > > http://doesnotexist.local and http://doesnotexist.local/icon > respectively > > to see make it pass. > > > > It seems that I triggered the creation of those entries in the > > RESOURCE_SERVER_RESOURCE table when > > I activated and deactivated the authz support for a client. > > > > I think this should be addressed in the migrations. There should be at > > least a note about that in the migration guides. > > It took me a while to find the table that contained the null values that > > were indirectly causing the migration to fail. > > > > Cheers, > > Thomas > > > > On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < > > Sebastian.Schuster at bosch-si.com> wrote: > > > > > Hi everybody, > > > > > > I just noticed that 4.2.1 contains a migration > > > (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column > from > > the > > > RESOURCE_SERVER_RESOURCE table and puts it into a separate table > > > RESOURCE_URIS. This table has a NOT NULL constraint on the new uri > column > > > (called VALUE). The accompanying data migration > > > AuthzResourceUseMoreURIs.java selects rows from the old table and > inserts > > > URIs it into the new. This fails for all resources that did not have a > > URI > > > before because of the NOT NULL constraint, for example for > > > Keycloak-internal resources like groups that don?t have a URI. > > > > > > Is this intended behavior? > > > > > > Best regards, > > > Sebastian > > > > > > Mit freundlichen Gr??en / Best regards > > > > > > Dr.-Ing. Sebastian Schuster > > > > > > Engineering and Support (INST/ESY1) > > > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > > > GERMANY | www.bosch-si.com > > > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > > > Sebastian.Schuster at bosch-si.com > > > > > > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > > > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: > Dr. > > > Stefan Ferber, Michael Hahn > > > > > > > > > > > > _______________________________________________ > > > keycloak-dev mailing list > > > keycloak-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From romain.poiffaut at elca.ch Wed Aug 8 01:50:59 2018 From: romain.poiffaut at elca.ch (Poiffaut Romain) Date: Wed, 8 Aug 2018 05:50:59 +0000 Subject: [keycloak-dev] SKIP_CACHE_STORE for Infinispan Message-ID: Hi, I am currently looking into the Infinispan cache usage in Keycloak. While reading the source code, I discovered InfinispanChangeLogBasedTransaction decorates the cache with SKIP_CACHE_STORE flag. This flag has been added to resolve KEYCLOAK-5656 regarding multiple DC. This fix has the side effect of preventing to persist the infos in an external cache store. What do you think about adding a mechanism to enable/disable this behavior according to the needs ? Best regards, Romain From Sebastian.Schuster at bosch-si.com Wed Aug 8 03:35:29 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Wed, 8 Aug 2018 07:35:29 +0000 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: References: Message-ID: <08a4c78dc9b04f61a766da4b1b605591@bosch-si.com> Thanks for fixing this so fast! Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn From: Pedro Igor Silva Sent: Mittwoch, 8. August 2018 00:31 To: Hynek Mlnarik Cc: Thomas Darimont ; keycloak-dev ; Schuster Sebastian (INST/ESY1) Subject: Re: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions Sent a PR: https://github.com/keycloak/keycloak/pull/5446. On Tue, Aug 7, 2018 at 3:08 PM, Hynek Mlnarik > wrote: Apologies for this oversight, this will be fixed in the next version. https://issues.jboss.org/browse/KEYCLOAK-8003 On Tue, Aug 7, 2018 at 7:00 PM Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hello, > > I was just bitten by this as well 3hours ago, but thankfully only in our > staging environment. We had only one entry > in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri and > icon_uri column. > This caused the migration to fail. In our prod env I there was no entry in > that table, so the migration went through. > As a quick fix in the staging env I just changed those uris to > http://doesnotexist.local and http://doesnotexist.local/icon respectively > to see make it pass. > > It seems that I triggered the creation of those entries in the > RESOURCE_SERVER_RESOURCE table when > I activated and deactivated the authz support for a client. > > I think this should be addressed in the migrations. There should be at > least a note about that in the migration guides. > It took me a while to find the table that contained the null values that > were indirectly causing the migration to fail. > > Cheers, > Thomas > > On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> wrote: > > > Hi everybody, > > > > I just noticed that 4.2.1 contains a migration > > (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column from > the > > RESOURCE_SERVER_RESOURCE table and puts it into a separate table > > RESOURCE_URIS. This table has a NOT NULL constraint on the new uri column > > (called VALUE). The accompanying data migration > > AuthzResourceUseMoreURIs.java selects rows from the old table and inserts > > URIs it into the new. This fails for all resources that did not have a > URI > > before because of the NOT NULL constraint, for example for > > Keycloak-internal resources like groups that don?t have a URI. > > > > Is this intended behavior? > > > > Best regards, > > Sebastian > > > > Mit freundlichen Gr??en / Best regards > > > > Dr.-Ing. Sebastian Schuster > > > > Engineering and Support (INST/ESY1) > > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > > GERMANY | www.bosch-si.com > > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > > Sebastian.Schuster at bosch-si.com> > > > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > > Stefan Ferber, Michael Hahn > > > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From l.lech at ringler.ch Wed Aug 8 03:36:54 2018 From: l.lech at ringler.ch (Lukasz Lech) Date: Wed, 8 Aug 2018 07:36:54 +0000 Subject: [keycloak-dev] Error while updating jboss/keycloak from old :latest to newer :latest Message-ID: <5E48B917000C984B86B77170F441903A119B54E4@exch.ringler.ch> Hello, I've started keycloak locally from docker image jboss/keycloak:latest Today I've pulled the image, and I've got the following error: Caused by: liquibase.exception.ValidationFailedException: Validation Failed: 1 change sets check sum META-INF/jpa-changelog-authz-4.0.0.CR1.xml::authz-4.0.0.CR1::psilva at redhat.com is now: 7:57960fc0b0f0dd0563ea6f8b2e4a1707 at liquibase.changelog.DatabaseChangeLog.validate(DatabaseChangeLog.java:206) at liquibase.Liquibase.listUnrunChangeSets(Liquibase.java:1139) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.keycloak.common.util.reflections.Reflections.invokeMethod(Reflections.java:379) The problem is, I'm not sure what was the :latest state I was using, it was some snapshot of 4.0.0-Beta, I've tried to start now :latest, :4.0.0.Final, and :4.0.0.Beta3, but neither of them can work with the old database. Does it mean, that my current database is unusable and I need to start from scratch or there is some way to update it? Should I never ever in the future use :latest image? Best regards, Lukasz Lech From hmlnarik at redhat.com Wed Aug 8 05:03:48 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Wed, 8 Aug 2018 11:03:48 +0200 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: <08a4c78dc9b04f61a766da4b1b605591@bosch-si.com> References: <08a4c78dc9b04f61a766da4b1b605591@bosch-si.com> Message-ID: The fix has been merged to latest master. On Wed, Aug 8, 2018 at 9:35 AM Schuster Sebastian (INST/ESY1) < Sebastian.Schuster at bosch-si.com> wrote: > Thanks for fixing this so fast! > > > > Best regards, > > Sebastian > > > > Mit freundlichen Gr??en / Best regards > > > *Dr.-Ing. Sebastian Schuster * > Engineering and Support (INST/ESY1) > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > GERMANY | www.bosch-si.com > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > Sebastian.Schuster at bosch-si.com > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > Stefan Ferber, Michael Hahn > > > > *From:* Pedro Igor Silva > *Sent:* Mittwoch, 8. August 2018 00:31 > *To:* Hynek Mlnarik > *Cc:* Thomas Darimont ; keycloak-dev < > keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> > *Subject:* Re: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs > fails with fine-grained admin permissions > > > > Sent a PR: https://github.com/keycloak/keycloak/pull/5446. > > > > On Tue, Aug 7, 2018 at 3:08 PM, Hynek Mlnarik wrote: > > Apologies for this oversight, this will be fixed in the next version. > > https://issues.jboss.org/browse/KEYCLOAK-8003 > > > On Tue, Aug 7, 2018 at 7:00 PM Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > > > Hello, > > > > I was just bitten by this as well 3hours ago, but thankfully only in our > > staging environment. We had only one entry > > in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri > and > > icon_uri column. > > This caused the migration to fail. In our prod env I there was no entry > in > > that table, so the migration went through. > > As a quick fix in the staging env I just changed those uris to > > http://doesnotexist.local and http://doesnotexist.local/icon > respectively > > to see make it pass. > > > > It seems that I triggered the creation of those entries in the > > RESOURCE_SERVER_RESOURCE table when > > I activated and deactivated the authz support for a client. > > > > I think this should be addressed in the migrations. There should be at > > least a note about that in the migration guides. > > It took me a while to find the table that contained the null values that > > were indirectly causing the migration to fail. > > > > Cheers, > > Thomas > > > > On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < > > Sebastian.Schuster at bosch-si.com> wrote: > > > > > Hi everybody, > > > > > > I just noticed that 4.2.1 contains a migration > > > (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column from > > the > > > RESOURCE_SERVER_RESOURCE table and puts it into a separate table > > > RESOURCE_URIS. This table has a NOT NULL constraint on the new uri > column > > > (called VALUE). The accompanying data migration > > > AuthzResourceUseMoreURIs.java selects rows from the old table and > inserts > > > URIs it into the new. This fails for all resources that did not have a > > URI > > > before because of the NOT NULL constraint, for example for > > > Keycloak-internal resources like groups that don?t have a URI. > > > > > > Is this intended behavior? > > > > > > Best regards, > > > Sebastian > > > > > > Mit freundlichen Gr??en / Best regards > > > > > > Dr.-Ing. Sebastian Schuster > > > > > > Engineering and Support (INST/ESY1) > > > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > > > GERMANY | www.bosch-si.com > > > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > > > Sebastian.Schuster at bosch-si.com > > > > > > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > > > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: > Dr. > > > Stefan Ferber, Michael Hahn > > > > > > > > > > > > _______________________________________________ > > > keycloak-dev mailing list > > > keycloak-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > From hmlnarik at redhat.com Wed Aug 8 08:35:06 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Wed, 8 Aug 2018 14:35:06 +0200 Subject: [keycloak-dev] Keycloak and SAML AudienceRestriction In-Reply-To: <1533206451.6676.1.camel@acutus.pro> References: <1533206451.6676.1.camel@acutus.pro> Message-ID: Thanks for raising this. We should implement a similar check that is for "aud" claim in OIDC and require strict validation. The actual value of the audience URI would be either realm URL or SAML endpoint URL within the realm. I've introduced a more general https://issues.jboss.org/browse/KEYCLOAK-8010 that will address this item. --Hynek On Thu, Aug 2, 2018 at 12:44 PM Dmitry Telegin
wrote: > Hi, > > It's been SAML time recently in keycloak-dev, so I won't be breaking > the trend... :) > > A customer tasked us with configuring Keycloak brokering to the 3rd > party SAML IdP. The IdP doesn't allow for SP metadata import, so the > values have to be configured manually, of which the two are mandatory, > namely Assertion Consumer Service URL and Audience (Entity ID). > > While things are crystal clear with ACS URL, there was some > misunderstanding with the Audience parameter. Assuming that it should > be equal to the EntityID of Keycloak (acting as an SP in this case), > we've put it there. After that, while reconfiguring for IdP-initiated > SSO, we have changed the ACS (the /clients/{url-name} suffix is > appended to it), but the question was what to do with Entity ID. By > experiment, we have determined that actually any non-empty value > worked. > > The situation is ambiguous, and we need to communicate it to the > customer somehow. The line in the docs "put any non-empty value" > smells fishy to me. I've found a technical explanation though; the > Audience (Entity ID) value ends up in the AudienceRestriction tag of > the SAML response. While Keycloak's SAML parser is aware of that tag, > it isn't processed in any way (ignored, in other words). > > Here's what the SAML spec says on AudienceRestriction: > > > Although a SAML relying party that is outside the audiences > > specified is capable of drawing conclusions from an assertion, the > > SAML asserting party explicitly makes no representation as to > > accuracy or trustworthiness to such a party... > > > > ...the element allows the SAML asserting > > party to state explicitly that no warranty is provided to such a > > party in a machine- and human-readable form. While there can be no > > guarantee that a court would uphold such a warrantyexclusion in every > > circumstance, the probability of upholding the warranty exclusion is > > considerably improved... > > http://docs.oasis-open.org/security/saml/v2.0/saml-core-2.0-os.pdf > > Nothing is said in the spec about if the AudienceRestriction check is > mandatory, so I'd suppose it is optional. Some SAML-enabled software > however implements strict checking, WebLogic being a well-known case. > > So it doesn't look like a defect or a security vulnerability, and > shouldn't pose any problems? Wanted to know the stance of the Keycloak > dev team on this. > > Thanks in advance! > Dmitry Telegin > CTO, Acutus s.r.o. > Keycloak Consulting and Training > > Pod lipami street 339/52, 130 00 Prague 3, Czech Republic > +42 (022) 888-30-71 > E-mail: info at acutus.pro > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From slaskawi at redhat.com Wed Aug 8 08:37:37 2018 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 8 Aug 2018 14:37:37 +0200 Subject: [keycloak-dev] Proposal - use Ant Plugin for generating JKS files In-Reply-To: References: Message-ID: On Fri, Jul 27, 2018 at 10:56 AM Hynek Mlnarik wrote: > My preference is programmatic approach too. See inline. > > On Thu, Jul 26, 2018 at 11:13 AM Sebastian Laskawiec > wrote: > >> Answering Stian's and Marek's email at the same time. >> >> Just FYI - I've created a ticket to track this work - >> https://issues.jboss.org/browse/KEYCLOAK-7940 >> >> On Wed, Jul 25, 2018 at 8:16 PM Marek Posolda >> wrote: >> >> > IMO Creating keystore files programatically as Peter suggested, looks >> > better than creating them through maven-antrun-plugin. It seems that >> > creating with maven will mean lots of the stuff in pom.xml files, >> > however generating programatically will probably allow some better >> > reusability (common utility class in the testsuite to easily generate >> > keystore in various locations). Also programatic generation will work >> > when running the test from Intellij Idea without previous maven build. >> > >> >> Perhaps I didn't make it clear enough - the JKS autogeneration script will >> mostly be executed in our test suite. I've just checked if you can run any >> of the tests with a clean repo (without invoking `mvn package` or `mvn >> install`) and it failed with [1]. So it seems you always need to execute >> Maven prior to invoking any tests from the IDE (at least from the >> Arquillian testsuite). >> > > The programmable approach Peter suggested looks very nice. However, I'm not >> sure if this is a good fit for us since we modify app servers during the >> build [2]. Those servers expect a keystore (and sometimes a truststore >> too) >> to be in certain places upon the server bootstrap. When the Arquillian >> testsuite if fired, it's simply to late to generate keystores on the fly. >> > > It is not too late. We modify the app servers during the build, and will > be only controlling their lifecycle in the testsuite, which is current work > in progress to be merged soon. [1] So one can always prepare the config for > the keystore and generate the keystore (if it does not exist) in Java to > appropriate place (ideally location would be set via system property). > I guess we have 3 different situations here that need to be addressed: 1) Adapter tests - there should be no problem here since they are more or less self-contained. So we might generate a keystore/truststore on the fly and include it in a Shrinkwrap package. 2) Tests that use private/public key in realm.json - also there should be no problem. The generator must be able to produce PEM entries. 3) Mutual TLS tests - that one is a bit problematic since for Mutual TLS we rely on Wildfly. And Wildfly expects JKS file to be available upon the start (the location is defined in standalone.xml). The easiest way to address #3 is to generate a keystore before the test, probably when building the WF-based test server. The alternative is to modify UndertowAppServer to enable Mutual TLS there and do all the Mutual TLS tests on pure Undertow. Even though this approach seems very attractive, there's one more piece there - Elytron with support of very nice things like key Certificate Revocation Lists etc. > > Adapter tests can then make use of the generated keystores since they > could be available to them via Arquillian-injected resources. That would > make tests that cope with encryption and/or signing simpler and I see quite > a benefit for at least SAML tests. > I totally agree. > > I believe this would be a neat follow-up on [1]. > > [1] https://issues.jboss.org/browse/KEYCLOAK-4407 > > [1] https://gist.github.com/slaskawi/b7e5f9973798345c1bef182ae8d8d55f >> [2] >> >> https://github.com/keycloak/keycloak/blob/54fcbf12b0740f092e2d9148fb4d06a634618839/testsuite/integration-arquillian/servers/app-server/jboss/pom.xml#L148 >> >> >> > Sebastian, yesterday I didn't mention on the call that we probably don't >> > test X509 authentication with IBM JDK. I recalled it just now :) The >> > thing is, that X509 tests are not executed during default build as they >> > require Wildfly (they don't work on embedded undertow) and also >> > phantomJS (no other browser is supported). So I am pretty sure they are >> > not executed during the various matrix jobs for various java versions >> > (Vasek Muzikar can correct me if I am wrong). Which means they probably >> > don't work on IBM JDK too. >> > >> >> In Infinispan we had exactly the same problem. Keystores generated by >> OpenJDK (or Oracle JDK) didn't work for IBM JDK. Therefore we added this >> [3] little trick. >> >> [3] >> >> https://github.com/keycloak/keycloak/pull/5410/files#diff-03eb8cfddae682425845854ec51a2d53R246 >> >> >> > >> > Marek >> > >> > >> > On 25/07/18 15:50, Stian Thorgersen wrote: >> > > Sounds like this could be a time-consuming task and the tests are >> working >> > > at the moment. >> >> >> I think I've solved the worst case - in some of the scenarios (like >> `DemoServletsAdapterTest#testOIDCUiLocalesParamForwarding`) we expect the >> certificate in keystore to be the same as the one from the realm JSON >> configuration. I might be wrong here (I'm still the "new guy") but the >> reason for this is that keystore.jks is used by the adapter whereas the >> auth server uses the certificate from json file. Those need to match. >> Therefore, I had to add a small piece of the code that does automatic >> replacement of PEM certificates in the configuration [4]. I could probably >> use some latest syntactic sugar (Optional#flatmap and Streams) but I think >> the old way is more readable in this case (but of course, comments are >> more >> than welcome, especially on the PR). >> >> [4] >> >> https://github.com/keycloak/keycloak/pull/5410/files#diff-207441f5cf5b6552db445afd0fe61492R28 >> >> >> > So not sure I see a need to start a bulk refactoring here. >> > >> >> Even though it might seem like a redundant thing to do, I would still >> prefer to replace all existing JKS entries. The main reason for this is >> that I'm a bit afraid that we'll end with a mixture of pushed JKS files >> and >> automatically generated ones. >> >> Let's make a deal. I'll look more into this today and will see how many of >> those I'll manage to replace. Maybe it won't be that bad after all... >> >> >> > > The json file you're mentioning has a cert in pem format in the json >> file >> > > that is then stored in the DB. This doesn't have anything to do with >> > > keystores/truststores. We expect Keycloak to be able to handle a cert >> > > provided in the json file (or through admin endpoints) and store it in >> > the >> > > DB. >> > > >> > > Now for SSL tests with the server itself I can see the value in what >> you >> > > are proposing. Keystore/truststore for Keycloak https connections and >> > > truststores for mutual SSL could probably be improved. However, it >> seems >> > to >> > > be working at the moment, so not sure this is required. >> >> >> >> > >> > > On Wed, 25 Jul 2018 at 15:30, Sebastian Laskawiec < >> slaskawi at redhat.com> >> > > wrote: >> > > >> > >> Thanks Darran and Peter for the insight! >> > >> >> > >> I've just noticed that generating JKS files is one thing but we will >> > also >> > >> need to pre-process RealmRepresentation and replace all >> private/public >> > keys >> > >> (like this one for example [1]) with values from >> keystores/truststores. >> > >> >> > >> At the moment I'm thinking about replacing [1] with something like >> this: >> > >> "jwt.credential.certificate" : >> > >> >> > >> >> > >> "certificate:keystore=classpath:keystore.jks,password=password,alias=secure-portal,type=pkcs12" >> > >> similarly to private keys (if needed): >> > >> "privateKey" : >> > >> >> > >> >> > >> "private-key:keystore=classpath:keystore.jks,password=password,alias=secure-portal,type=pkcs12" >> > >> >> > >> Later on, while converting JSON to RealmRepresentation I plan to >> > traverse >> > >> all nodes and replace replace those which start with private-key or >> > >> certificate. >> > >> >> > >> Let me know if you have a better idea. >> > >> >> > >> [1] >> > >> >> > >> >> > >> https://github.com/keycloak/keycloak/blob/master/testsuite/integration-arquillian/tests/base/src/test/resources/adapter-test/demorealm.json#L231 >> > >> >> > >> On Wed, Jul 25, 2018 at 2:07 PM Peter Skopek >> > wrote: >> > >> >> > >>> Hi Sebastian, >> > >>> >> > >>> I think that generation of Keystore on the fly is the only way to >> go. >> > >>> You can use Java Keystore API generate it entirely from Java code >> (no >> > >>> need to run maven). >> > >>> >> > >>> Here [1] is an example of utility class used in wildfly testsuite to >> > >>> generate Vault (basically just keystore). >> > >>> It will be easy to strip vault related stuff and use it in our >> > testsuite. >> > >>> >> > >>> Note that there is one more drawback of storing generated JKS files, >> > >>> which is that in some cases they are not binary compatible between >> > >>> different JDK (particularly Oracle and IBM). >> > >>> >> > >>> HTH, >> > >>> Peter >> > >>> >> > >>> [1] >> > >>> >> > >> >> > >> https://github.com/wildfly/wildfly/blob/master/testsuite/shared/src/main/java/org/jboss/as/test/integration/security/common/VaultHandler.java >> > >>> On Wed, Jul 25, 2018 at 1:23 PM Sebastian Laskawiec < >> > slaskawi at redhat.com >> > >>> >> > >>> wrote: >> > >>>> Hey, >> > >>>> >> > >>>> Together with Sebi we are working on Certificate-based >> authentication >> > >> for >> > >>>> the clients. Our work will require adding at least 2-3 keystores to >> > the >> > >>>> codebase with different DNs and I think this might be a good >> > >> opportunity >> > >>> to >> > >>>> revisit the way we handle JKS files in the tests. >> > >>>> >> > >>>> Currently we push JKS files directly into our repo, which has a >> couple >> > >> of >> > >>>> drawbacks: >> > >>>> - it is hard to figure out what's inside the JKS, it requires >> looking >> > >> up >> > >>>> for password (usually in some JSON configuration file or hardcoded >> in >> > >> the >> > >>>> test) and using keytool (or some similar one) to explore its >> content. >> > >>>> - It is not git-friendly. Every time we update JKS content we >> > >> effectively >> > >>>> store another binary file (git doesn't understand binary file >> changes >> > >> and >> > >>>> can not diff it). >> > >>>> - we use many different naming and password schemes in our tests. >> > >>>> - it is hard to migrate all keystores to pkcs12 at the same time >> (JKS >> > >>>> format is deprecated) [0] >> > >>>> >> > >>>> I believe most of the issues could be addressed by generating JKS >> > files >> > >>> on >> > >>>> the fly - during the build. In Infinispan we did it with Maven Ant >> > >> Plugin >> > >>>> [1]. I already created a very limited POC for Keycloak and you can >> > >> check >> > >>> it >> > >>>> out here [2]. Unfortunately, the process of reverse engineering all >> > >> those >> > >>>> files is quite time-consuming, so I would like to know your opinion >> > >>> before >> > >>>> moving on. >> > >>>> >> > >>>> Of course, generating JKS files on the fly has some drawbacks: >> > >>>> - It increases build time (~1s per keytool invocation, and we >> probably >> > >>> will >> > >>>> have more than 30 of them). >> > >>>> - it makes testing from IDE a bit harder, you need to run Maven and >> > >>> process >> > >>>> test resources before doing anything. A common workaround is to use >> > >> `mvn >> > >>>> clean install -DskipTests` and then opening your IntelliJ. >> > >>>> >> > >>>> Please let me know what you think. >> > >>>> >> > >>>> Thanks, >> > >>>> Sebastian >> > >>>> >> > >>>> [0] >> > >>>> >> > >> >> > >> https://blogs.oracle.com/jtc/jdk9-keytool-transitions-default-keystore-to-pkcs12 >> > >>>> [1] >> > >>>> >> > >> >> > >> https://github.com/infinispan/infinispan/blob/master/server/integration/testsuite/pom.xml#L460 >> > >>>> [2] https://github.com/keycloak/keycloak/pull/5410 >> > >>>> _______________________________________________ >> > >>>> keycloak-dev mailing list >> > >>>> keycloak-dev at lists.jboss.org >> > >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> _______________________________________________ >> > >> keycloak-dev mailing list >> > >> keycloak-dev at lists.jboss.org >> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> >> > > _______________________________________________ >> > > keycloak-dev mailing list >> > > keycloak-dev at lists.jboss.org >> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> > >> > >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > From dt at acutus.pro Wed Aug 8 15:00:55 2018 From: dt at acutus.pro (Dmitry Telegin) Date: Wed, 08 Aug 2018 22:00:55 +0300 Subject: [keycloak-dev] Authentication flow deep/shallow copy ambiguity In-Reply-To: <92ae2968-f8bd-0e67-db38-3b2f7f84d600@redhat.com> References: <1533562460.13393.8.camel@acutus.pro> <92ae2968-f8bd-0e67-db38-3b2f7f84d600@redhat.com> Message-ID: <1533754855.10533.3.camel@acutus.pro> https://issues.jboss.org/browse/KEYCLOAK-8011 Not sure if I have enough time for the fix now, maybe in September. Dmitry On Tue, 2018-08-07 at 08:10 +0200, Marek Posolda wrote: > Hi Dmitry, > > yes this looks like a bug to me. Feel free to create JIRA. Even better? > PR with the test ;) (There are some tests in the? > org.keycloak.testsuite.admin.authentication package and I hope the test? > for this could be added here. > > Marek > > On 06/08/18 15:34, Dmitry Telegin wrote: > > Hi, > > > > When copying authentication flow where some authenticators have been > > configured, I'd expect deep copy, i.e. including authenticator configs. > > But that's not the case, here are the steps to reproduce: > > > > - create Copy of Browser Flow; > > - configure Identity Provider Redirector with "foo" IdP; > > - create Copy of Copy of Browser flow; > > - go to Identity Provider Redirector config, change "foo" to "bar"; > > - go back to Copy of Browser Flow. The redirector config will change to > > "bar". > > > > Is this a bug? If not, I think this should be documented, because it's > > not obvious and can cause confusion. > > > > Thanks, > > Dmitry > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > From mposolda at redhat.com Wed Aug 8 15:58:52 2018 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 8 Aug 2018 21:58:52 +0200 Subject: [keycloak-dev] Authentication flow deep/shallow copy ambiguity In-Reply-To: <1533754855.10533.3.camel@acutus.pro> References: <1533562460.13393.8.camel@acutus.pro> <92ae2968-f8bd-0e67-db38-3b2f7f84d600@redhat.com> <1533754855.10533.3.camel@acutus.pro> Message-ID: Cool, Thanks! Marek On 08/08/18 21:00, Dmitry Telegin wrote: > https://issues.jboss.org/browse/KEYCLOAK-8011 > > Not sure if I have enough time for the fix now, maybe in September. > > Dmitry > > On Tue, 2018-08-07 at 08:10 +0200, Marek Posolda wrote: >> Hi Dmitry, >> >> yes this looks like a bug to me. Feel free to create JIRA. Even better >> PR with the test ;) (There are some tests in the >> org.keycloak.testsuite.admin.authentication package and I hope the test >> for this could be added here. >> >> Marek >> >> On 06/08/18 15:34, Dmitry Telegin wrote: >>> Hi, >>> >>> When copying authentication flow where some authenticators have been >>> configured, I'd expect deep copy, i.e. including authenticator configs. >>> But that's not the case, here are the steps to reproduce: >>> >>> - create Copy of Browser Flow; >>> - configure Identity Provider Redirector with "foo" IdP; >>> - create Copy of Copy of Browser flow; >>> - go to Identity Provider Redirector config, change "foo" to "bar"; >>> - go back to Copy of Browser Flow. The redirector config will change to >>> "bar". >>> >>> Is this a bug? If not, I think this should be documented, because it's >>> not obvious and can cause confusion. >>> >>> Thanks, >>> Dmitry >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> From mposolda at redhat.com Thu Aug 9 03:11:06 2018 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 9 Aug 2018 09:11:06 +0200 Subject: [keycloak-dev] Error while updating jboss/keycloak from old :latest to newer :latest In-Reply-To: <5E48B917000C984B86B77170F441903A119B54E4@exch.ringler.ch> References: <5E48B917000C984B86B77170F441903A119B54E4@exch.ringler.ch> Message-ID: <061ac76c-e8f5-ead2-61bc-c9134df6e96d@redhat.com> On 08/08/18 09:36, Lukasz Lech wrote: > Hello, > > I've started keycloak locally from docker image jboss/keycloak:latest > > Today I've pulled the image, and I've got the following error: > > Caused by: liquibase.exception.ValidationFailedException: Validation Failed: > 1 change sets check sum > META-INF/jpa-changelog-authz-4.0.0.CR1.xml::authz-4.0.0.CR1::psilva at redhat.com is now: 7:57960fc0b0f0dd0563ea6f8b2e4a1707 > > at liquibase.changelog.DatabaseChangeLog.validate(DatabaseChangeLog.java:206) > at liquibase.Liquibase.listUnrunChangeSets(Liquibase.java:1139) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.keycloak.common.util.reflections.Reflections.invokeMethod(Reflections.java:379) > The problem is, I'm not sure what was the :latest state I was using, it was some snapshot of 4.0.0-Beta, > I've tried to start now :latest, :4.0.0.Final, and :4.0.0.Beta3, but neither of them can work with the old database. > > Does it mean, that my current database is unusable and I need to start from scratch or there is some way to update it? Not sure if there is still a way to update it now... The error you're seeing is triggered by Liquibase (DB migration tool used by Keycloak) as it sees the update in some changelog, which you had already migrated in your DB. In general yes, in your production DB, you should ideally never use "some snapshot of 4.0.0-Beta" as there is a chance that DB would be already "partially" migrated to 4.0.0 and liquibase won't be able to reliably detect the state in which exactly the DB is. So the possibility is to either: - start from scratch - Try to fix DB manually. You can inspect Keycloak changelog files and state of your DB and see whether you can try to apply some remaining changes manually or revert to the state before 4.0.0 and then re-trigger the migration. This may not be trivial to do, but it's helpful if you don't have any other way and you don't want to loose the data > > Should I never ever in the future use :latest image? Probably not. It's not good that :latest points to some snapshot version. This is maybe something, we should improve on KC side. But still, it's always better to use some "known" version instead of latest. I personally wouldn't use "latest" for anything in production. Marek > > Best regards, > Lukasz Lech > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From mposolda at redhat.com Thu Aug 9 03:16:23 2018 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 9 Aug 2018 09:16:23 +0200 Subject: [keycloak-dev] SKIP_CACHE_STORE for Infinispan In-Reply-To: References: Message-ID: On 08/08/18 07:50, Poiffaut Romain wrote: > Hi, > > I am currently looking into the Infinispan cache usage in Keycloak. > While reading the source code, I discovered InfinispanChangeLogBasedTransaction decorates the cache with SKIP_CACHE_STORE flag. This flag has been added to resolve KEYCLOAK-5656 regarding multiple DC. > > This fix has the side effect of preventing to persist the infos in an external cache store. > > What do you think about adding a mechanism to enable/disable this behavior according to the needs ? Yes, I think it will be good. Feel free to create JIRA for this. ATM we're not testing with cacheStore/cacheLoader enabled, so the regression is possible as you pointed. For fixing this, I think we don't even need to introduce some new configuration option. We can probably detect the flags based on the fact if the cacheStore/cacheLoader is remote-store (then use flags) or some other store like JDBC , File or something else (then not use flags). BTV. There is possible workaround for this issue, that you configure Keycloak with the remote-store, which will point to some external infinispan/JDG server. Then on the caches on the JDG side, you will configure the JDBC or File store. I know this is not ideal as you will need the JDG server just to workaround this limitation. Also no guarantee everything will work as expected as we didn't test this setup (I just guess that it will work as a workaround). Marek > > Best regards, > Romain > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From thomas.darimont at googlemail.com Thu Aug 9 06:42:59 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Thu, 9 Aug 2018 12:42:59 +0200 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: References: <08a4c78dc9b04f61a766da4b1b605591@bosch-si.com> Message-ID: Awesome thanks Hynek et al., Are you planning a new 4.2.x or rather a 4.3.x release with the fix? Cheers, Thomas Hynek Mlnarik schrieb am Mi., 8. Aug. 2018, 11:03: > The fix has been merged to latest master. > > On Wed, Aug 8, 2018 at 9:35 AM Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> wrote: > >> Thanks for fixing this so fast! >> >> >> >> Best regards, >> >> Sebastian >> >> >> >> Mit freundlichen Gr??en / Best regards >> >> >> *Dr.-Ing. Sebastian Schuster * >> Engineering and Support (INST/ESY1) >> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> GERMANY | www.bosch-si.com >> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> Sebastian.Schuster at bosch-si.com >> >> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >> Dr. Stefan Ferber, Michael Hahn >> >> >> >> *From:* Pedro Igor Silva >> *Sent:* Mittwoch, 8. August 2018 00:31 >> *To:* Hynek Mlnarik >> *Cc:* Thomas Darimont ; keycloak-dev < >> keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) < >> Sebastian.Schuster at bosch-si.com> >> *Subject:* Re: [keycloak-dev] Migration to 4.2.1 extracting >> RESOURCE_URIs fails with fine-grained admin permissions >> >> >> >> Sent a PR: https://github.com/keycloak/keycloak/pull/5446. >> >> >> >> On Tue, Aug 7, 2018 at 3:08 PM, Hynek Mlnarik >> wrote: >> >> Apologies for this oversight, this will be fixed in the next version. >> >> https://issues.jboss.org/browse/KEYCLOAK-8003 >> >> >> On Tue, Aug 7, 2018 at 7:00 PM Thomas Darimont < >> thomas.darimont at googlemail.com> wrote: >> >> > Hello, >> > >> > I was just bitten by this as well 3hours ago, but thankfully only in our >> > staging environment. We had only one entry >> > in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri >> and >> > icon_uri column. >> > This caused the migration to fail. In our prod env I there was no entry >> in >> > that table, so the migration went through. >> > As a quick fix in the staging env I just changed those uris to >> > http://doesnotexist.local and http://doesnotexist.local/icon >> respectively >> > to see make it pass. >> > >> > It seems that I triggered the creation of those entries in the >> > RESOURCE_SERVER_RESOURCE table when >> > I activated and deactivated the authz support for a client. >> > >> > I think this should be addressed in the migrations. There should be at >> > least a note about that in the migration guides. >> > It took me a while to find the table that contained the null values that >> > were indirectly causing the migration to fail. >> > >> > Cheers, >> > Thomas >> > >> > On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < >> > Sebastian.Schuster at bosch-si.com> wrote: >> > >> > > Hi everybody, >> > > >> > > I just noticed that 4.2.1 contains a migration >> > > (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column >> from >> > the >> > > RESOURCE_SERVER_RESOURCE table and puts it into a separate table >> > > RESOURCE_URIS. This table has a NOT NULL constraint on the new uri >> column >> > > (called VALUE). The accompanying data migration >> > > AuthzResourceUseMoreURIs.java selects rows from the old table and >> inserts >> > > URIs it into the new. This fails for all resources that did not have a >> > URI >> > > before because of the NOT NULL constraint, for example for >> > > Keycloak-internal resources like groups that don?t have a URI. >> > > >> > > Is this intended behavior? >> > > >> > > Best regards, >> > > Sebastian >> > > >> > > Mit freundlichen Gr??en / Best regards >> > > >> > > Dr.-Ing. Sebastian Schuster >> > > >> > > Engineering and Support (INST/ESY1) >> > > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> > > GERMANY | www.bosch-si.com >> > > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> > > Sebastian.Schuster at bosch-si.com> Sebastian.Schuster at bosch-si.com> >> > > >> > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 >> B >> > > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >> Dr. >> > > Stefan Ferber, Michael Hahn >> > > >> > > >> > > >> > > _______________________________________________ >> > > keycloak-dev mailing list >> > > keycloak-dev at lists.jboss.org >> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > _______________________________________________ >> > keycloak-dev mailing list >> > keycloak-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > From mposolda at redhat.com Thu Aug 9 13:04:40 2018 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 9 Aug 2018 19:04:40 +0200 Subject: [keycloak-dev] Migration to 4.2.1 extracting RESOURCE_URIs fails with fine-grained admin permissions In-Reply-To: References: <08a4c78dc9b04f61a766da4b1b605591@bosch-si.com> Message-ID: <1fb38c20-1551-6e7a-759c-f02ba6518caa@redhat.com> Hi Thomas, rather a 4.3.x release with the fix. Marek On 09/08/18 12:42, Thomas Darimont wrote: > Awesome thanks Hynek et al., > > Are you planning a new 4.2.x or rather a 4.3.x release with the fix? > > Cheers, > Thomas > > Hynek Mlnarik schrieb am Mi., 8. Aug. 2018, 11:03: > >> The fix has been merged to latest master. >> >> On Wed, Aug 8, 2018 at 9:35 AM Schuster Sebastian (INST/ESY1) < >> Sebastian.Schuster at bosch-si.com> wrote: >> >>> Thanks for fixing this so fast! >>> >>> >>> >>> Best regards, >>> >>> Sebastian >>> >>> >>> >>> Mit freundlichen Gr??en / Best regards >>> >>> >>> *Dr.-Ing. Sebastian Schuster * >>> Engineering and Support (INST/ESY1) >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >>> GERMANY | www.bosch-si.com >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >>> Sebastian.Schuster at bosch-si.com >>> >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >>> Dr. Stefan Ferber, Michael Hahn >>> >>> >>> >>> *From:* Pedro Igor Silva >>> *Sent:* Mittwoch, 8. August 2018 00:31 >>> *To:* Hynek Mlnarik >>> *Cc:* Thomas Darimont ; keycloak-dev < >>> keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) < >>> Sebastian.Schuster at bosch-si.com> >>> *Subject:* Re: [keycloak-dev] Migration to 4.2.1 extracting >>> RESOURCE_URIs fails with fine-grained admin permissions >>> >>> >>> >>> Sent a PR: https://github.com/keycloak/keycloak/pull/5446. >>> >>> >>> >>> On Tue, Aug 7, 2018 at 3:08 PM, Hynek Mlnarik >>> wrote: >>> >>> Apologies for this oversight, this will be fixed in the next version. >>> >>> https://issues.jboss.org/browse/KEYCLOAK-8003 >>> >>> >>> On Tue, Aug 7, 2018 at 7:00 PM Thomas Darimont < >>> thomas.darimont at googlemail.com> wrote: >>> >>>> Hello, >>>> >>>> I was just bitten by this as well 3hours ago, but thankfully only in our >>>> staging environment. We had only one entry >>>> in the RESOURCE_SERVER_RESOURCE table that had a null value in the uri >>> and >>>> icon_uri column. >>>> This caused the migration to fail. In our prod env I there was no entry >>> in >>>> that table, so the migration went through. >>>> As a quick fix in the staging env I just changed those uris to >>>> http://doesnotexist.local and http://doesnotexist.local/icon >>> respectively >>>> to see make it pass. >>>> >>>> It seems that I triggered the creation of those entries in the >>>> RESOURCE_SERVER_RESOURCE table when >>>> I activated and deactivated the authz support for a client. >>>> >>>> I think this should be addressed in the migrations. There should be at >>>> least a note about that in the migration guides. >>>> It took me a while to find the table that contained the null values that >>>> were indirectly causing the migration to fail. >>>> >>>> Cheers, >>>> Thomas >>>> >>>> On Tue, Aug 7, 2018 at 5:25 PM Schuster Sebastian (INST/ESY1) < >>>> Sebastian.Schuster at bosch-si.com> wrote: >>>> >>>>> Hi everybody, >>>>> >>>>> I just noticed that 4.2.1 contains a migration >>>>> (jpa-changelog-authz-4.2.0.Final.xml) that extracts the URI column >>> from >>>> the >>>>> RESOURCE_SERVER_RESOURCE table and puts it into a separate table >>>>> RESOURCE_URIS. This table has a NOT NULL constraint on the new uri >>> column >>>>> (called VALUE). The accompanying data migration >>>>> AuthzResourceUseMoreURIs.java selects rows from the old table and >>> inserts >>>>> URIs it into the new. This fails for all resources that did not have a >>>> URI >>>>> before because of the NOT NULL constraint, for example for >>>>> Keycloak-internal resources like groups that don?t have a URI. >>>>> >>>>> Is this intended behavior? >>>>> >>>>> Best regards, >>>>> Sebastian >>>>> >>>>> Mit freundlichen Gr??en / Best regards >>>>> >>>>> Dr.-Ing. Sebastian Schuster >>>>> >>>>> Engineering and Support (INST/ESY1) >>>>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >>>>> GERMANY | www.bosch-si.com >>>>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >>>>> Sebastian.Schuster at bosch-si.com>> Sebastian.Schuster at bosch-si.com> >>>>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 >>> B >>>>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >>> Dr. >>>>> Stefan Ferber, Michael Hahn >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From l.lech at ringler.ch Fri Aug 10 04:09:10 2018 From: l.lech at ringler.ch (Lukasz Lech) Date: Fri, 10 Aug 2018 08:09:10 +0000 Subject: [keycloak-dev] Keycloak 4.2.1 - Counter based OTP Message-ID: <5E48B917000C984B86B77170F441903A119B662D@exch.ringler.ch> Hello, I'm using Keycloak 4.2.1.Final (docker image). I've defined OTP policy: Counter based, SHA1, Look Ahead 100, initial counter 1 I've tested number of digits both 6 and 8 I've tried to use Free OTP, for 6 digits Google Auth. I've deleted authentication, scanned QR once again, tested a few next codes, not only the first. Each time I get the message, the one-time code is invalid. Do counter based policy works at all in Keycloak 4.2.1? I don't see what I'm doing wrong... Best regards, Lukasz Lech From hmlnarik at redhat.com Fri Aug 10 04:58:20 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Fri, 10 Aug 2018 10:58:20 +0200 Subject: [keycloak-dev] Keycloak 4.2.1 - Counter based OTP In-Reply-To: <5E48B917000C984B86B77170F441903A119B662D@exch.ringler.ch> References: <5E48B917000C984B86B77170F441903A119B662D@exch.ringler.ch> Message-ID: This mailing list is dedicated to discussions around Keycloak development. Please resend your question to keycloak-user mailing list [1] which is a place for any questions related to installing, setting up, and using Keycloak. [1] https://lists.jboss.org/mailman/listinfo/keycloak-user On Fri, Aug 10, 2018 at 10:10 AM Lukasz Lech wrote: > Hello, > > I'm using Keycloak 4.2.1.Final (docker image). > > I've defined OTP policy: Counter based, SHA1, Look Ahead 100, initial > counter 1 > I've tested number of digits both 6 and 8 > I've tried to use Free OTP, for 6 digits Google Auth. > I've deleted authentication, scanned QR once again, tested a few next > codes, not only the first. > > Each time I get the message, the one-time code is invalid. > > Do counter based policy works at all in Keycloak 4.2.1? I don't see what > I'm doing wrong... > > Best regards, > Lukasz Lech > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From bruno at abstractj.org Mon Aug 13 09:23:45 2018 From: bruno at abstractj.org (Bruno Oliveira) Date: Mon, 13 Aug 2018 10:23:45 -0300 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis Message-ID: <20180813132345.GA28783@abstractj.org> Good morning, Last week Pedro submitted a PR to the Node.js adapter, but the build is failing because it depends on the changes from Keycloak server master branch. Today we download the latest stable release from Keycloak to run the integration tests for this adapter. I would like to change it and follow the same approach from the Quickstarts, which means clone/build/run Keycloak server from master. In this way, we can always it test against the latest changes. Thoughts? -- abstractj From jambo_mcd at yahoo.co.uk Mon Aug 13 09:47:30 2018 From: jambo_mcd at yahoo.co.uk (Jamie McDowell) Date: Mon, 13 Aug 2018 13:47:30 +0000 (UTC) Subject: [keycloak-dev] Multiple password policies References: <1356792787.9540666.1534168050614.ref@mail.yahoo.com> Message-ID: <1356792787.9540666.1534168050614@mail.yahoo.com> Hi, Can you have multiple password policies on the same realm for different OU's?? We have Keycloak set up federating to an OpenLDAP server. On the LDAP server we have 2 OU's, 1 for users and the other for service accounts - Both of these need to have different passwords such as length and complexity.? We have the password policy defined on the OpenLDAP. Has anyone configured this before? Regards, Jamie From sblanc at redhat.com Mon Aug 13 10:01:20 2018 From: sblanc at redhat.com (Sebastien Blanc) Date: Mon, 13 Aug 2018 16:01:20 +0200 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: <20180813132345.GA28783@abstractj.org> References: <20180813132345.GA28783@abstractj.org> Message-ID: +1 , we should always rely on master. On Mon, Aug 13, 2018 at 3:23 PM, Bruno Oliveira wrote: > Good morning, > > Last week Pedro submitted a PR to the Node.js adapter, but the build is > failing because it depends on the changes from Keycloak server master > branch. > > Today we download the latest stable release from Keycloak to run the > integration tests for this adapter. I would like to change it and follow > the same approach from the Quickstarts, which means clone/build/run > Keycloak server from master. In this way, we can always it test against > the latest changes. > > Thoughts? > > > -- > > abstractj > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From johannes at kodet.no Mon Aug 13 17:44:58 2018 From: johannes at kodet.no (Johannes Knutsen) Date: Mon, 13 Aug 2018 23:44:58 +0200 Subject: [keycloak-dev] Locale cookie vs ui_locales priority Message-ID: Currently the LocaleHelper#getUserLocale chooses locale priority based on the following order: 1. kc_locale query parameter 2. Locale cookie 3. User profile 4. ui_locales query parameter 5. Accept-Language http header We are building clients which uses the ui_locales parameter to select localization of the login to match the locale of the client. Since the locale cookie has a higher priority than the ui_locales query parameter, the ui_locales query parameter is ignored for any auth requests with a kc_locale cookie present. 1. Do you see any implications of prioritizing the ui_locales parameter above the locale cookie when resolving locale? 2. Are there reasons why the ui_locales does not update the locale cookie value the same way kc_locale parameter does it? Best regards, Johannes Knutsen From sthorger at redhat.com Tue Aug 14 03:16:23 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 14 Aug 2018 07:16:23 +0000 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: The way at least it was supposed to work was that the Locale cookie would only be set if a user has explicitly changed the locale. That way we can remember a users preference even when the user is logged out. Are you seeing ui_locales not taking affect without a user having used the locale drop-down? On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen wrote: > Currently the LocaleHelper#getUserLocale chooses locale priority based > on the following order: > > 1. kc_locale query parameter > 2. Locale cookie > 3. User profile > 4. ui_locales query parameter > 5. Accept-Language http header > > We are building clients which uses the ui_locales parameter to select > localization of the login to match the locale of the client. > Since the locale cookie has a higher priority than the ui_locales > query parameter, the ui_locales query parameter is ignored for any > auth requests with a kc_locale cookie present. > > 1. Do you see any implications of prioritizing the ui_locales > parameter above the locale cookie when resolving locale? > 2. Are there reasons why the ui_locales does not update the locale > cookie value the same way kc_locale parameter does it? > > Best regards, > Johannes Knutsen > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Tue Aug 14 03:20:30 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 14 Aug 2018 07:20:30 +0000 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: References: <20180813132345.GA28783@abstractj.org> Message-ID: I agree it should test with something at least close to master. Question is should all the repos build Keycloak themselves? We could have Docker images automatically built/hosted on Docker hub? There's already a nightly (seems to be broken atm, but that's fixable). That may be more efficient than building, especially considering Maven dependencies. On Mon, 13 Aug 2018 at 14:13, Sebastien Blanc wrote: > +1 , we should always rely on master. > > On Mon, Aug 13, 2018 at 3:23 PM, Bruno Oliveira > wrote: > > > Good morning, > > > > Last week Pedro submitted a PR to the Node.js adapter, but the build is > > failing because it depends on the changes from Keycloak server master > > branch. > > > > Today we download the latest stable release from Keycloak to run the > > integration tests for this adapter. I would like to change it and follow > > the same approach from the Quickstarts, which means clone/build/run > > Keycloak server from master. In this way, we can always it test against > > the latest changes. > > > > Thoughts? > > > > > > -- > > > > abstractj > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From vmuzikar at redhat.com Tue Aug 14 05:22:06 2018 From: vmuzikar at redhat.com (Vaclav Muzikar) Date: Tue, 14 Aug 2018 11:22:06 +0200 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: <20180813132345.GA28783@abstractj.org> References: <20180813132345.GA28783@abstractj.org> Message-ID: +1 We've got a nightly CI job testing Node.js adapter against upstream but running it in Travis (with each PR) would make more sense. On Mon, Aug 13, 2018 at 3:40 PM Bruno Oliveira wrote: > Good morning, > > Last week Pedro submitted a PR to the Node.js adapter, but the build is > failing because it depends on the changes from Keycloak server master > branch. > > Today we download the latest stable release from Keycloak to run the > integration tests for this adapter. I would like to change it and follow > the same approach from the Quickstarts, which means clone/build/run > Keycloak server from master. In this way, we can always it test against > the latest changes. > > Thoughts? > > > -- > > abstractj > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- V?clav Muzik?? Quality Engineer Keycloak / Red Hat Single Sign-On Red Hat Czech s.r.o. From alistair.doswald at elca.ch Tue Aug 14 05:53:53 2018 From: alistair.doswald at elca.ch (Doswald Alistair) Date: Tue, 14 Aug 2018 09:53:53 +0000 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project Message-ID: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Hello, I just wanted to let this mailing list know that for the Cloudtrust project (https://github.com/cloudtrust), we have developed a certain number modules for Keycloak. These are currently compatible with the version 3.4.3.Final of Keycloak, but we will make them compatible with Keycloak 4.X (where X will be the latest sub-version of Keycloak when we start working on this) as soon as we can. These modules are: * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an implementation of the WS-Federation protocol for keycloak. This allows to select the WS-Federation protocol for Keycloak clients and for identity brokers. * keycloak-authorization (https://github.com/cloudtrust/keycloak-authorization): this module allows the use of the client authorization system to prevent a user which is authenticated in a Keycloak realm to access a given client. It works no matter which protocol is used, and without the client having to support any extra protocol. Note: this solution is a bit hacky, but necessary for one of our use-cases. * keycloak-client-mappers (https://github.com/cloudtrust/keycloak-client-mappers): a module for adding any mappers that we might need that are not yet part of Keycloak. Currently only contains a JavaScript mapper for SAML, analogous to the OIDC script mapper. I've noticed that there's an open issue for this feature (https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could submit this code not as a module but a solution to the issue. * keycloak-export (https://github.com/cloudtrust/keycloak-export): a module adding an endpoint to fully export a realm while Keycloak is still running (no need for restarts!). Cheers, Alistair PS: I mailed this to both dev and user mailing lists as I believe it may interest members of both mailing lists. However, upon sending to the dev mailing list the first time it bounced. This is the second attempt. From johannes at kodet.no Tue Aug 14 06:56:26 2018 From: johannes at kodet.no (Johannes Knutsen) Date: Tue, 14 Aug 2018 12:56:26 +0200 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: It currently works the way you describe it. The ui_locales parameter is taking effect as long as the locale cookie isn't set. However, the logic implies that you always expect the same user to authenticate and our preference is that the user should get a login page matching the locale of the client which sends the user to the login page, if the client specified preferred locales as an ui_locales parameter. Using the locale cookie to override the ui_locales parameter makes the user experience inconsistent when moving from a client with a locale different from the last logged in user's preferred locale. A proposed change is to change locale priority as follows: 1. kc_locale query parameter. This updates the user's locale and locale cookie. 2. user profile locale if a user is authenticated. This updates the locale cookie. 3. ui_locales query parameter. If ui_locales contains the locale cookie value, return it. Otherwise, return the first locale available from ui_locales 4. locale cookie 5. accept language http header Is this change something you would consider merging if I create a pull request? Any other adjustments or thoughts regarding such a change? - Johannes On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen wrote: > The way at least it was supposed to work was that the Locale cookie would > only be set if a user has explicitly changed the locale. That way we can > remember a users preference even when the user is logged out. > > Are you seeing ui_locales not taking affect without a user having used the > locale drop-down? > > On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen wrote: >> >> Currently the LocaleHelper#getUserLocale chooses locale priority based >> on the following order: >> >> 1. kc_locale query parameter >> 2. Locale cookie >> 3. User profile >> 4. ui_locales query parameter >> 5. Accept-Language http header >> >> We are building clients which uses the ui_locales parameter to select >> localization of the login to match the locale of the client. >> Since the locale cookie has a higher priority than the ui_locales >> query parameter, the ui_locales query parameter is ignored for any >> auth requests with a kc_locale cookie present. >> >> 1. Do you see any implications of prioritizing the ui_locales >> parameter above the locale cookie when resolving locale? >> 2. Are there reasons why the ui_locales does not update the locale >> cookie value the same way kc_locale parameter does it? >> >> Best regards, >> Johannes Knutsen >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev From hmlnarik at redhat.com Tue Aug 14 08:33:34 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Tue, 14 Aug 2018 14:33:34 +0200 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: References: <20180813132345.GA28783@abstractj.org> Message-ID: Is nightly enough? Consider keycloak repo breaks due to some change and the quickstarts cannot be built until this is fixed. In nightly, that would delay the development to the next day. My vote is to either build Keycloak from master like Bruno suggested or have a way documented to rebuild the "latest" image (regardless of "nightly" name) anytime on demand to enable dependent changes to be developed quickly. --Hynek On Tue, Aug 14, 2018 at 11:34 AM Vaclav Muzikar wrote: > +1 > > We've got a nightly CI job testing Node.js adapter against upstream but > running it in Travis (with each PR) would make more sense. > > On Mon, Aug 13, 2018 at 3:40 PM Bruno Oliveira > wrote: > > > Good morning, > > > > Last week Pedro submitted a PR to the Node.js adapter, but the build is > > failing because it depends on the changes from Keycloak server master > > branch. > > > > Today we download the latest stable release from Keycloak to run the > > integration tests for this adapter. I would like to change it and follow > > the same approach from the Quickstarts, which means clone/build/run > > Keycloak server from master. In this way, we can always it test against > > the latest changes. > > > > Thoughts? > > > > > > -- > > > > abstractj > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > -- > V?clav Muzik?? > Quality Engineer > Keycloak / Red Hat Single Sign-On > Red Hat Czech s.r.o. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From bruno at abstractj.org Tue Aug 14 09:08:01 2018 From: bruno at abstractj.org (Bruno Oliveira) Date: Tue, 14 Aug 2018 10:08:01 -0300 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: References: <20180813132345.GA28783@abstractj.org> Message-ID: Let's do this, I will change it to always build Keycloak from master and later we can think about docker images. I think Docker images would be better, at the same time, I'm not sure how fix the scenario where feature A on Node.js adapter, depends on Feature B merged on Keycloak master and both have to be merged/tested in the same day. Unless we do something like Hynek suggested. On Tue, Aug 14, 2018 at 9:33 AM Hynek Mlnarik wrote: > > Is nightly enough? > > Consider keycloak repo breaks due to some change and the quickstarts cannot be built until this is fixed. In nightly, that would delay the development to the next day. > > My vote is to either build Keycloak from master like Bruno suggested or have a way documented to rebuild the "latest" image (regardless of "nightly" name) anytime on demand to enable dependent changes to be developed quickly. > > --Hynek > > On Tue, Aug 14, 2018 at 11:34 AM Vaclav Muzikar wrote: >> >> +1 >> >> We've got a nightly CI job testing Node.js adapter against upstream but >> running it in Travis (with each PR) would make more sense. >> >> On Mon, Aug 13, 2018 at 3:40 PM Bruno Oliveira wrote: >> >> > Good morning, >> > >> > Last week Pedro submitted a PR to the Node.js adapter, but the build is >> > failing because it depends on the changes from Keycloak server master >> > branch. >> > >> > Today we download the latest stable release from Keycloak to run the >> > integration tests for this adapter. I would like to change it and follow >> > the same approach from the Quickstarts, which means clone/build/run >> > Keycloak server from master. In this way, we can always it test against >> > the latest changes. >> > >> > Thoughts? >> > >> > >> > -- >> > >> > abstractj >> > _______________________________________________ >> > keycloak-dev mailing list >> > keycloak-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> >> >> -- >> V?clav Muzik?? >> Quality Engineer >> Keycloak / Red Hat Single Sign-On >> Red Hat Czech s.r.o. >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev -- - abstractj From sthorger at redhat.com Tue Aug 14 14:24:33 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 14 Aug 2018 20:24:33 +0200 Subject: [keycloak-dev] Truststore SPI Message-ID: Was thinking some more about how to share certificate functionality between user authentication and client authentication. Potentially also to SAML. We could add a Truststore SPI. The TruststoreProvider interface would have something like the following methods (quick mock stuff): * boolean verifyCert(CertificateMatcher matcher) * Set listExpiring() * void trustCert(Cert cert) * void removeTrustedCert(String id) * Cert getSavedCert(String id) * void removeSavedCert(String id) * void saveCert(Cert cert) We'd have a default built-in TruststoreProvider. Potentially delegating some stuff to Elytron, but it would have to store certs in the db. It would deal with checking if cert is still valid, call cert validation like the user cert does, etc.. For user and client authentication we'd use verifyCert primarily. We'd have something like UserCertMatcher.create()... and ClientCertMatcher.create()... In the admin console we could add an option to add trusted certs (not sure how that'd work in a cluster). For SAML clients we'd delegate storing certs to the Truststore SPI instead of storing the cert data alongside the client. The truststore would also add some metadata in the db about certs so we can easily query for expiring certs. From sthorger at redhat.com Tue Aug 14 14:32:01 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 14 Aug 2018 20:32:01 +0200 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: The most common case is that the same user uses the browser. So if the user has actively changed the locale in Keycloak it should stay like that. I would actually suggest that if the user has changed the locale in Keycloak, you should have the locale added to the token so the application can update it's locale. That would result in all applications in the realm using the same locale for the user. Now if we change it to what you are suggesting you can end up with situations where the first login page is displayed in one language, then once the user has entered username/email it would end up switching to the locale associated with the user. If you really don't like that behaviour I would suggest we either introduce a LocaleSelectorSPI where you can change the logic with a custom provider. Alternatively, we could potentially add an option on the realm to be able to disable the locale selector on Keycloak, which means that the user wouldn't be able to change the locale and ui_locales would always take precedence. In either case I think it's best to keep the current behaviour as the default, with some way of being able to change it. On Tue, 14 Aug 2018 at 12:56, Johannes Knutsen wrote: > It currently works the way you describe it. The ui_locales parameter > is taking effect as long as the locale cookie isn't set. However, the > logic implies that you always expect the same user to authenticate and > our preference is that the user should get a login page matching the > locale of the client which sends the user to the login page, if the > client specified preferred locales as an ui_locales parameter. > Using the locale cookie to override the ui_locales parameter makes the > user experience inconsistent when moving from a client with a locale > different from the last logged in user's preferred locale. > > A proposed change is to change locale priority as follows: > 1. kc_locale query parameter. This updates the user's locale and locale > cookie. > 2. user profile locale if a user is authenticated. This updates the > locale cookie. > 3. ui_locales query parameter. If ui_locales contains the locale > cookie value, return it. Otherwise, return the first locale available > from ui_locales > 4. locale cookie > 5. accept language http header > > Is this change something you would consider merging if I create a pull > request? Any other adjustments or thoughts regarding such a change? > > - Johannes > > On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen > wrote: > > The way at least it was supposed to work was that the Locale cookie would > > only be set if a user has explicitly changed the locale. That way we can > > remember a users preference even when the user is logged out. > > > > Are you seeing ui_locales not taking affect without a user having used > the > > locale drop-down? > > > > On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen > wrote: > >> > >> Currently the LocaleHelper#getUserLocale chooses locale priority based > >> on the following order: > >> > >> 1. kc_locale query parameter > >> 2. Locale cookie > >> 3. User profile > >> 4. ui_locales query parameter > >> 5. Accept-Language http header > >> > >> We are building clients which uses the ui_locales parameter to select > >> localization of the login to match the locale of the client. > >> Since the locale cookie has a higher priority than the ui_locales > >> query parameter, the ui_locales query parameter is ignored for any > >> auth requests with a kc_locale cookie present. > >> > >> 1. Do you see any implications of prioritizing the ui_locales > >> parameter above the locale cookie when resolving locale? > >> 2. Are there reasons why the ui_locales does not update the locale > >> cookie value the same way kc_locale parameter does it? > >> > >> Best regards, > >> Johannes Knutsen > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > From thomas.darimont at googlemail.com Tue Aug 14 14:58:20 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 14 Aug 2018 20:58:20 +0200 Subject: [keycloak-dev] Fine-grained permissions along hierarchy paths Message-ID: Hello, I have a realm with nested groups that denotes a hierarchical corporate structure. /corp -/org --/branch1 ---/divsion1 ----/team1 ----/team2 ---/divsion2 ----/team3 ----/team4 --/branch2 -/infra ... Users belong to one particular group along the /corp/org subtree, but might also be members of one or more groups from a different subtree, e.g., /corp/infra. Is it possible to have dedicated admin users at /corp, /branchX, /divisionX level who can only view and manage the users from their group or subtree with an admin-console scoped to a fixed realm? admin-console scoped to group-hierarchy-demo realm: http://localhost:8080/auth/admin/group-hierarchy-demo/console/#/realms/group-hierarchy-demo/users If a user logs in as divsion1-admin-user, he should only be able to see and manage the users beneath the path (/corp/org/branch1/division1/*). Does the fine-grained permission system already support use cases like this? Cheers, Thomas From thomas.darimont at googlemail.com Tue Aug 14 15:26:21 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 14 Aug 2018 21:26:21 +0200 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: <4dd6baaaf6654cb79441536c2118f798@elca.ch> References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Message-ID: Hello Alistair, those are IMHO awesome modules thanks for sharing :) btw. you also have a handy go Keycloak client :) https://github.com/cloudtrust/keycloak-client Regarding SAML ScriptMapper (KEYCLOAK-5520) I think it totally makes sense to integrate that into Keycloak directly. I was onto writing that myself but then priorities changed..., but your implementation looks quite good already :) I'm pretty sure that if you get the tests running inside the Keycloak test-suite the Keycloak team would be happy to discuss/merge your PR. Cheers, Thomas Am Di., 14. Aug. 2018 um 12:04 Uhr schrieb Doswald Alistair < alistair.doswald at elca.ch>: > Hello, > > I just wanted to let this mailing list know that for the Cloudtrust > project (https://github.com/cloudtrust), we have developed a certain > number modules for Keycloak. These are currently compatible with the > version 3.4.3.Final of Keycloak, but we will make them compatible with > Keycloak 4.X (where X will be the latest sub-version of Keycloak when we > start working on this) as soon as we can. These modules are: > > * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an > implementation of the WS-Federation protocol for keycloak. This allows to > select the WS-Federation protocol for Keycloak clients and for identity > brokers. > > * keycloak-authorization ( > https://github.com/cloudtrust/keycloak-authorization): this module allows > the use of the client authorization system to prevent a user which is > authenticated in a Keycloak realm to access a given client. It works no > matter which protocol is used, and without the client having to support any > extra protocol. Note: this solution is a bit hacky, but necessary for one > of our use-cases. > > * keycloak-client-mappers ( > https://github.com/cloudtrust/keycloak-client-mappers): a module for > adding any mappers that we might need that are not yet part of Keycloak. > Currently only contains a JavaScript mapper for SAML, analogous to the OIDC > script mapper. I've noticed that there's an open issue for this feature ( > https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could > submit this code not as a module but a solution to the issue. > > * keycloak-export (https://github.com/cloudtrust/keycloak-export): a > module adding an endpoint to fully export a realm while Keycloak is still > running (no need for restarts!). > > Cheers, > > Alistair > > PS: I mailed this to both dev and user mailing lists as I believe it may > interest members of both mailing lists. However, upon sending to the dev > mailing list the first time it bounced. This is the second attempt. > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From Sebastian.Schuster at bosch-si.com Wed Aug 15 03:44:18 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Wed, 15 Aug 2018 07:44:18 +0000 Subject: [keycloak-dev] Fine-grained permissions along hierarchy paths In-Reply-To: References: Message-ID: Hi Thomas, I think this should work. You will just have to enable permissions for the groups /corp, /branchX, /divisionX and create matching policies and assign the scopes view-members and manage-members. If a user is a member of one of the subgroups, the permissions defined on the parent groups still kick in. You just need to be aware that listing all users does not work as expected, see https://issues.jboss.org/browse/KEYCLOAK-7950. If you navigate via the groups, you should be fine... I am just not sure what you mean by "admin console scoped to a fixed realm". All of this only works on the same realm, other realms are completely separate things... Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch?Software Innovations?GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn -----Original Message----- From: keycloak-dev-bounces at lists.jboss.org On Behalf Of Thomas Darimont Sent: Dienstag, 14. August 2018 20:58 To: keycloak-dev ; keycloak-user Subject: [keycloak-dev] Fine-grained permissions along hierarchy paths Hello, I have a realm with nested groups that denotes a hierarchical corporate structure. /corp -/org --/branch1 ---/divsion1 ----/team1 ----/team2 ---/divsion2 ----/team3 ----/team4 --/branch2 -/infra ... Users belong to one particular group along the /corp/org subtree, but might also be members of one or more groups from a different subtree, e.g., /corp/infra. Is it possible to have dedicated admin users at /corp, /branchX, /divisionX level who can only view and manage the users from their group or subtree with an admin-console scoped to a fixed realm? admin-console scoped to group-hierarchy-demo realm: http://localhost:8080/auth/admin/group-hierarchy-demo/console/#/realms/group-hierarchy-demo/users If a user logs in as divsion1-admin-user, he should only be able to see and manage the users beneath the path (/corp/org/branch1/division1/*). Does the fine-grained permission system already support use cases like this? Cheers, Thomas _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From ssilvert at redhat.com Wed Aug 15 08:55:37 2018 From: ssilvert at redhat.com (Stan Silvert) Date: Wed, 15 Aug 2018 08:55:37 -0400 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: <5a48253b-10e8-7dab-839b-ca95d569c7b1@redhat.com> On 8/14/2018 2:32 PM, Stian Thorgersen wrote: > The most common case is that the same user uses the browser. So if the user > has actively changed the locale in Keycloak it should stay like that. I > would actually suggest that if the user has changed the locale in Keycloak, > you should have the locale added to the token so the application can update > it's locale. That would result in all applications in the realm using the > same locale for the user. > > Now if we change it to what you are suggesting you can end up with > situations where the first login page is displayed in one language, then > once the user has entered username/email it would end up switching to the > locale associated with the user. > > If you really don't like that behaviour I would suggest we either introduce > a LocaleSelectorSPI where you can change the logic with a custom provider. > Alternatively, we could potentially add an option on the realm to be able > to disable the locale selector on Keycloak, which means that the user > wouldn't be able to change the locale and ui_locales would always take > precedence. In either case I think it's best to keep the current behaviour > as the default, with some way of being able to change it. It's easy for Johannes to disable the locale selector.? Just remove it from the template.ftl file.? Or to be cleaner, create a new template.ftl for your realm and override the one in the base realm. A LocaleSelectorSPI would be really nice.? I just wonder how much demand there would be for this. > > On Tue, 14 Aug 2018 at 12:56, Johannes Knutsen wrote: > >> It currently works the way you describe it. The ui_locales parameter >> is taking effect as long as the locale cookie isn't set. However, the >> logic implies that you always expect the same user to authenticate and >> our preference is that the user should get a login page matching the >> locale of the client which sends the user to the login page, if the >> client specified preferred locales as an ui_locales parameter. >> Using the locale cookie to override the ui_locales parameter makes the >> user experience inconsistent when moving from a client with a locale >> different from the last logged in user's preferred locale. >> >> A proposed change is to change locale priority as follows: >> 1. kc_locale query parameter. This updates the user's locale and locale >> cookie. >> 2. user profile locale if a user is authenticated. This updates the >> locale cookie. >> 3. ui_locales query parameter. If ui_locales contains the locale >> cookie value, return it. Otherwise, return the first locale available >> from ui_locales >> 4. locale cookie >> 5. accept language http header >> >> Is this change something you would consider merging if I create a pull >> request? Any other adjustments or thoughts regarding such a change? >> >> - Johannes >> >> On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen >> wrote: >>> The way at least it was supposed to work was that the Locale cookie would >>> only be set if a user has explicitly changed the locale. That way we can >>> remember a users preference even when the user is logged out. >>> >>> Are you seeing ui_locales not taking affect without a user having used >> the >>> locale drop-down? >>> >>> On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen >> wrote: >>>> Currently the LocaleHelper#getUserLocale chooses locale priority based >>>> on the following order: >>>> >>>> 1. kc_locale query parameter >>>> 2. Locale cookie >>>> 3. User profile >>>> 4. ui_locales query parameter >>>> 5. Accept-Language http header >>>> >>>> We are building clients which uses the ui_locales parameter to select >>>> localization of the login to match the locale of the client. >>>> Since the locale cookie has a higher priority than the ui_locales >>>> query parameter, the ui_locales query parameter is ignored for any >>>> auth requests with a kc_locale cookie present. >>>> >>>> 1. Do you see any implications of prioritizing the ui_locales >>>> parameter above the locale cookie when resolving locale? >>>> 2. Are there reasons why the ui_locales does not update the locale >>>> cookie value the same way kc_locale parameter does it? >>>> >>>> Best regards, >>>> Johannes Knutsen >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From johannes at kodet.no Wed Aug 15 11:13:17 2018 From: johannes at kodet.no (Johannes Knutsen) Date: Wed, 15 Aug 2018 17:13:17 +0200 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: We do already add the user's locale to the token and will try to update the clients locale based on the token. However, since the common case is that the same user uses the browser on each login, the ui_locales parameter is typically just ignored and gives very little value as it is currently implemented. For the common cases where you would like to have the default behavior, the ui_locales parameter is simply not sent to the auth request. When the ui_locales parameter is provided from the client, I still think it should be used and override the locale cookie if the locale value is not present in ui_locales. A LocaleSelectorSPI does make sense. Not sure of the demand, however localization is typically something I guess people would never agree upon. Thus, giving the option to customize it makes sense. A custom implementation could for example use a locale cookie global to an organization's root domain instead of the realm specific locale cookie. I will try to have a look at the amount of work required to extract such a SPI. On Tue, Aug 14, 2018 at 8:32 PM, Stian Thorgersen wrote: > The most common case is that the same user uses the browser. So if the user > has actively changed the locale in Keycloak it should stay like that. I > would actually suggest that if the user has changed the locale in Keycloak, > you should have the locale added to the token so the application can update > it's locale. That would result in all applications in the realm using the > same locale for the user. > > Now if we change it to what you are suggesting you can end up with > situations where the first login page is displayed in one language, then > once the user has entered username/email it would end up switching to the > locale associated with the user. > > If you really don't like that behaviour I would suggest we either introduce > a LocaleSelectorSPI where you can change the logic with a custom provider. > Alternatively, we could potentially add an option on the realm to be able to > disable the locale selector on Keycloak, which means that the user wouldn't > be able to change the locale and ui_locales would always take precedence. In > either case I think it's best to keep the current behaviour as the default, > with some way of being able to change it. > > On Tue, 14 Aug 2018 at 12:56, Johannes Knutsen wrote: >> >> It currently works the way you describe it. The ui_locales parameter >> is taking effect as long as the locale cookie isn't set. However, the >> logic implies that you always expect the same user to authenticate and >> our preference is that the user should get a login page matching the >> locale of the client which sends the user to the login page, if the >> client specified preferred locales as an ui_locales parameter. >> Using the locale cookie to override the ui_locales parameter makes the >> user experience inconsistent when moving from a client with a locale >> different from the last logged in user's preferred locale. >> >> A proposed change is to change locale priority as follows: >> 1. kc_locale query parameter. This updates the user's locale and locale >> cookie. >> 2. user profile locale if a user is authenticated. This updates the >> locale cookie. >> 3. ui_locales query parameter. If ui_locales contains the locale >> cookie value, return it. Otherwise, return the first locale available >> from ui_locales >> 4. locale cookie >> 5. accept language http header >> >> Is this change something you would consider merging if I create a pull >> request? Any other adjustments or thoughts regarding such a change? >> >> - Johannes >> >> On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen >> wrote: >> > The way at least it was supposed to work was that the Locale cookie >> > would >> > only be set if a user has explicitly changed the locale. That way we can >> > remember a users preference even when the user is logged out. >> > >> > Are you seeing ui_locales not taking affect without a user having used >> > the >> > locale drop-down? >> > >> > On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen >> > wrote: >> >> >> >> Currently the LocaleHelper#getUserLocale chooses locale priority based >> >> on the following order: >> >> >> >> 1. kc_locale query parameter >> >> 2. Locale cookie >> >> 3. User profile >> >> 4. ui_locales query parameter >> >> 5. Accept-Language http header >> >> >> >> We are building clients which uses the ui_locales parameter to select >> >> localization of the login to match the locale of the client. >> >> Since the locale cookie has a higher priority than the ui_locales >> >> query parameter, the ui_locales query parameter is ignored for any >> >> auth requests with a kc_locale cookie present. >> >> >> >> 1. Do you see any implications of prioritizing the ui_locales >> >> parameter above the locale cookie when resolving locale? >> >> 2. Are there reasons why the ui_locales does not update the locale >> >> cookie value the same way kc_locale parameter does it? >> >> >> >> Best regards, >> >> Johannes Knutsen >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev From johannes at kodet.no Wed Aug 15 12:13:28 2018 From: johannes at kodet.no (Johannes Knutsen) Date: Wed, 15 Aug 2018 18:13:28 +0200 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: I did some quick work on extracting an LocaleSelectorSPI. Please have a look at https://github.com/keycloak/keycloak/compare/master...knutz3n:feature/locale-spi. Thoughts? - Johannes On Tue, Aug 14, 2018 at 8:32 PM, Stian Thorgersen wrote: > The most common case is that the same user uses the browser. So if the user > has actively changed the locale in Keycloak it should stay like that. I > would actually suggest that if the user has changed the locale in Keycloak, > you should have the locale added to the token so the application can update > it's locale. That would result in all applications in the realm using the > same locale for the user. > > Now if we change it to what you are suggesting you can end up with > situations where the first login page is displayed in one language, then > once the user has entered username/email it would end up switching to the > locale associated with the user. > > If you really don't like that behaviour I would suggest we either introduce > a LocaleSelectorSPI where you can change the logic with a custom provider. > Alternatively, we could potentially add an option on the realm to be able to > disable the locale selector on Keycloak, which means that the user wouldn't > be able to change the locale and ui_locales would always take precedence. In > either case I think it's best to keep the current behaviour as the default, > with some way of being able to change it. > > On Tue, 14 Aug 2018 at 12:56, Johannes Knutsen wrote: >> >> It currently works the way you describe it. The ui_locales parameter >> is taking effect as long as the locale cookie isn't set. However, the >> logic implies that you always expect the same user to authenticate and >> our preference is that the user should get a login page matching the >> locale of the client which sends the user to the login page, if the >> client specified preferred locales as an ui_locales parameter. >> Using the locale cookie to override the ui_locales parameter makes the >> user experience inconsistent when moving from a client with a locale >> different from the last logged in user's preferred locale. >> >> A proposed change is to change locale priority as follows: >> 1. kc_locale query parameter. This updates the user's locale and locale >> cookie. >> 2. user profile locale if a user is authenticated. This updates the >> locale cookie. >> 3. ui_locales query parameter. If ui_locales contains the locale >> cookie value, return it. Otherwise, return the first locale available >> from ui_locales >> 4. locale cookie >> 5. accept language http header >> >> Is this change something you would consider merging if I create a pull >> request? Any other adjustments or thoughts regarding such a change? >> >> - Johannes >> >> On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen >> wrote: >> > The way at least it was supposed to work was that the Locale cookie >> > would >> > only be set if a user has explicitly changed the locale. That way we can >> > remember a users preference even when the user is logged out. >> > >> > Are you seeing ui_locales not taking affect without a user having used >> > the >> > locale drop-down? >> > >> > On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen >> > wrote: >> >> >> >> Currently the LocaleHelper#getUserLocale chooses locale priority based >> >> on the following order: >> >> >> >> 1. kc_locale query parameter >> >> 2. Locale cookie >> >> 3. User profile >> >> 4. ui_locales query parameter >> >> 5. Accept-Language http header >> >> >> >> We are building clients which uses the ui_locales parameter to select >> >> localization of the login to match the locale of the client. >> >> Since the locale cookie has a higher priority than the ui_locales >> >> query parameter, the ui_locales query parameter is ignored for any >> >> auth requests with a kc_locale cookie present. >> >> >> >> 1. Do you see any implications of prioritizing the ui_locales >> >> parameter above the locale cookie when resolving locale? >> >> 2. Are there reasons why the ui_locales does not update the locale >> >> cookie value the same way kc_locale parameter does it? >> >> >> >> Best regards, >> >> Johannes Knutsen >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev From warren.weeder at deque.com Wed Aug 15 23:18:42 2018 From: warren.weeder at deque.com (Warren Weeder) Date: Wed, 15 Aug 2018 23:18:42 -0400 Subject: [keycloak-dev] Recent changes breaking keycloak.. Message-ID: It appears that breaking changes were in the 'latest' version released about 6 hours ago. Container based on latest image fails with 'hostname not set'. Using version 4.2.1.final instead of latest, everything works fine. Note: I am using mssql jdbc driver. This is a working build which only recently breaks as of the most recent 'latest' update. Prior to that, it worked fine building off latest, and it works fine now that I have forced version '4.2.1.Final'. thanks Warren From bdawidow at redhat.com Thu Aug 16 00:02:15 2018 From: bdawidow at redhat.com (Boleslaw Dawidowicz) Date: Thu, 16 Aug 2018 06:02:15 +0200 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: <4dd6baaaf6654cb79441536c2118f798@elca.ch> References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Message-ID: Consider making a PR so all are listed in Extensions section on the website. WSFed already is: https://www.keycloak.org/extensions.html https://github.com/keycloak/keycloak.github.io/blob/master/extensions.html On Tue, Aug 14, 2018 at 12:05 PM Doswald Alistair wrote: > Hello, > > I just wanted to let this mailing list know that for the Cloudtrust > project (https://github.com/cloudtrust), we have developed a certain > number modules for Keycloak. These are currently compatible with the > version 3.4.3.Final of Keycloak, but we will make them compatible with > Keycloak 4.X (where X will be the latest sub-version of Keycloak when we > start working on this) as soon as we can. These modules are: > > * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an > implementation of the WS-Federation protocol for keycloak. This allows to > select the WS-Federation protocol for Keycloak clients and for identity > brokers. > > * keycloak-authorization ( > https://github.com/cloudtrust/keycloak-authorization): this module allows > the use of the client authorization system to prevent a user which is > authenticated in a Keycloak realm to access a given client. It works no > matter which protocol is used, and without the client having to support any > extra protocol. Note: this solution is a bit hacky, but necessary for one > of our use-cases. > > * keycloak-client-mappers ( > https://github.com/cloudtrust/keycloak-client-mappers): a module for > adding any mappers that we might need that are not yet part of Keycloak. > Currently only contains a JavaScript mapper for SAML, analogous to the OIDC > script mapper. I've noticed that there's an open issue for this feature ( > https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could > submit this code not as a module but a solution to the issue. > > * keycloak-export (https://github.com/cloudtrust/keycloak-export): a > module adding an endpoint to fully export a realm while Keycloak is still > running (no need for restarts!). > > Cheers, > > Alistair > > PS: I mailed this to both dev and user mailing lists as I believe it may > interest members of both mailing lists. However, upon sending to the dev > mailing list the first time it bounced. This is the second attempt. > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Thu Aug 16 04:21:28 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 16 Aug 2018 10:21:28 +0200 Subject: [keycloak-dev] Recent changes breaking keycloak.. In-Reply-To: References: Message-ID: Please provide more information. Steps to reproduce, log output, etc.. On Thu, 16 Aug 2018 at 09:56, Warren Weeder wrote: > It appears that breaking changes were in the 'latest' version released > about 6 hours ago. Container based on latest image fails with 'hostname not > set'. > > Using version 4.2.1.final instead of latest, everything works fine. > > Note: I am using mssql jdbc driver. This is a working build which only > recently breaks as of the most recent 'latest' update. Prior to that, it > worked fine building off latest, and it works fine now that I have forced > version '4.2.1.Final'. > > > > > thanks > Warren > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Thu Aug 16 04:30:13 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 16 Aug 2018 10:30:13 +0200 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: <4dd6baaaf6654cb79441536c2118f798@elca.ch> References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Message-ID: On Tue, 14 Aug 2018 at 12:03, Doswald Alistair wrote: > Hello, > > I just wanted to let this mailing list know that for the Cloudtrust > project (https://github.com/cloudtrust), we have developed a certain > number modules for Keycloak. These are currently compatible with the > version 3.4.3.Final of Keycloak, but we will make them compatible with > Keycloak 4.X (where X will be the latest sub-version of Keycloak when we > start working on this) as soon as we can. These modules are: > > * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an > implementation of the WS-Federation protocol for keycloak. This allows to > select the WS-Federation protocol for Keycloak clients and for identity > brokers. > This is great and pleased that you are maintaining this. We simply don't have the capacity to have this incorporated into Keycloak as we lack expertise on WS-Fed and have to many other things to maintain. > > * keycloak-authorization ( > https://github.com/cloudtrust/keycloak-authorization): this module allows > the use of the client authorization system to prevent a user which is > authenticated in a Keycloak realm to access a given client. It works no > matter which protocol is used, and without the client having to support any > extra protocol. Note: this solution is a bit hacky, but necessary for one > of our use-cases. > Sounds interesting. Can you write some more details about this? In the past I've considered if we should have something like required roles to use to a client. That would by not allowing a user to authenticate with a client, but also could prevent adding audience for a service the user isn't allowed to access. Could perhaps take it one step further and prevent client scopes that users are not allowed to access. > > * keycloak-client-mappers ( > https://github.com/cloudtrust/keycloak-client-mappers): a module for > adding any mappers that we might need that are not yet part of Keycloak. > Currently only contains a JavaScript mapper for SAML, analogous to the OIDC > script mapper. I've noticed that there's an open issue for this feature ( > https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could > submit this code not as a module but a solution to the issue. > SAML JavaScript protocol mapper would be great. If you submit with tests and docs we'd love to have this in. > > * keycloak-export (https://github.com/cloudtrust/keycloak-export): a > module adding an endpoint to fully export a realm while Keycloak is still > running (no need for restarts!). > Can you write some more details about this? In the past we've considered this, but not added it mainly for a few reasons, including: * What happens if there are changes while doing the export, we where thinking we'd have to pause the server from receiving requests during this export * We don't want a REST endpoint that can return credentials and other secrets. Then there's also the size of the export. It would have to store the export on disc, but then what about clustering. One potential would be to develop a tool that can be executed from anywhere that has access to the db. It would use Keycloak code base, but instead of starting a server it would allow things like running an export from the db, migrate the db, etc. > > Cheers, > > Alistair > > PS: I mailed this to both dev and user mailing lists as I believe it may > interest members of both mailing lists. However, upon sending to the dev > mailing list the first time it bounced. This is the second attempt. > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Thu Aug 16 04:33:53 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 16 Aug 2018 10:33:53 +0200 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: References: <20180813132345.GA28783@abstractj.org> Message-ID: How about this: * Rebuild Docker image for every PR. We can probably do this with a webhook or somethin'. * Write a script that can be shared to all projects that need a Keycloak server. It would check the latest Docker hub image and see if it matches master. If it doesn't it would build the image locally. Then it would start the Keycloak server. Could use the same script for Node.js, Generic Adapter, QuickStarts, etc.. On Tue, 14 Aug 2018 at 15:10, Bruno Oliveira wrote: > Let's do this, I will change it to always build Keycloak from master > and later we can think about docker images. I think Docker images > would be better, at the same time, I'm not sure how fix the scenario > where feature A on Node.js adapter, depends on Feature B merged on > Keycloak master and both have to be merged/tested in the same day. > > Unless we do something like Hynek suggested. > On Tue, Aug 14, 2018 at 9:33 AM Hynek Mlnarik wrote: > > > > Is nightly enough? > > > > Consider keycloak repo breaks due to some change and the quickstarts > cannot be built until this is fixed. In nightly, that would delay the > development to the next day. > > > > My vote is to either build Keycloak from master like Bruno suggested or > have a way documented to rebuild the "latest" image (regardless of > "nightly" name) anytime on demand to enable dependent changes to be > developed quickly. > > > > --Hynek > > > > On Tue, Aug 14, 2018 at 11:34 AM Vaclav Muzikar > wrote: > >> > >> +1 > >> > >> We've got a nightly CI job testing Node.js adapter against upstream but > >> running it in Travis (with each PR) would make more sense. > >> > >> On Mon, Aug 13, 2018 at 3:40 PM Bruno Oliveira > wrote: > >> > >> > Good morning, > >> > > >> > Last week Pedro submitted a PR to the Node.js adapter, but the build > is > >> > failing because it depends on the changes from Keycloak server master > >> > branch. > >> > > >> > Today we download the latest stable release from Keycloak to run the > >> > integration tests for this adapter. I would like to change it and > follow > >> > the same approach from the Quickstarts, which means clone/build/run > >> > Keycloak server from master. In this way, we can always it test > against > >> > the latest changes. > >> > > >> > Thoughts? > >> > > >> > > >> > -- > >> > > >> > abstractj > >> > _______________________________________________ > >> > keycloak-dev mailing list > >> > keycloak-dev at lists.jboss.org > >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > >> > >> > >> -- > >> V?clav Muzik?? > >> Quality Engineer > >> Keycloak / Red Hat Single Sign-On > >> Red Hat Czech s.r.o. > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > - abstractj > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From sthorger at redhat.com Thu Aug 16 06:15:18 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 16 Aug 2018 12:15:18 +0200 Subject: [keycloak-dev] Keycloak 4.3.0.Final is out Message-ID: https://www.keycloak.org/docs/latest/release_notes/index.html#keycloak-4-3-0-final From bruno at abstractj.org Thu Aug 16 09:12:22 2018 From: bruno at abstractj.org (Bruno Oliveira) Date: Thu, 16 Aug 2018 10:12:22 -0300 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: References: <20180813132345.GA28783@abstractj.org> Message-ID: On Thu, Aug 16, 2018 at 5:34 AM Stian Thorgersen wrote: > > How about this: > > * Rebuild Docker image for every PR. We can probably do this with a webhook or somethin'. If we think about 3 repositories or more getting new PRs. This might become a bit expensive, don't you think? I'd suggest another alternative, but that would require a bot to trigger the docker build on demand. This would be the flow proposed: 1. PR submitted and tested against the nightly build 2. PR failed the author add a comment like @keycloak-bot build-keycloak-server rerun 3. The bot check if user has the rights for it, trigger the build of the docker image and run the tests again This is just a suggestion which I understand, requires some time. But this is the best which I can come up with. Or we can try what you suggested and see how it goes. > * Write a script that can be shared to all projects that need a Keycloak server. It would check the latest Docker hub image and see if it matches master. If it doesn't it would build the image locally. Then it would start the Keycloak server. Could use the same script for Node.js, Generic Adapter, QuickStarts, etc.. +1 Once we reach an agreement on this, with whatever approach we decided. We can file a Jira. > > On Tue, 14 Aug 2018 at 15:10, Bruno Oliveira wrote: >> >> Let's do this, I will change it to always build Keycloak from master >> and later we can think about docker images. I think Docker images >> would be better, at the same time, I'm not sure how fix the scenario >> where feature A on Node.js adapter, depends on Feature B merged on >> Keycloak master and both have to be merged/tested in the same day. >> >> Unless we do something like Hynek suggested. >> On Tue, Aug 14, 2018 at 9:33 AM Hynek Mlnarik wrote: >> > >> > Is nightly enough? >> > >> > Consider keycloak repo breaks due to some change and the quickstarts cannot be built until this is fixed. In nightly, that would delay the development to the next day. >> > >> > My vote is to either build Keycloak from master like Bruno suggested or have a way documented to rebuild the "latest" image (regardless of "nightly" name) anytime on demand to enable dependent changes to be developed quickly. >> > >> > --Hynek >> > >> > On Tue, Aug 14, 2018 at 11:34 AM Vaclav Muzikar wrote: >> >> >> >> +1 >> >> >> >> We've got a nightly CI job testing Node.js adapter against upstream but >> >> running it in Travis (with each PR) would make more sense. >> >> >> >> On Mon, Aug 13, 2018 at 3:40 PM Bruno Oliveira wrote: >> >> >> >> > Good morning, >> >> > >> >> > Last week Pedro submitted a PR to the Node.js adapter, but the build is >> >> > failing because it depends on the changes from Keycloak server master >> >> > branch. >> >> > >> >> > Today we download the latest stable release from Keycloak to run the >> >> > integration tests for this adapter. I would like to change it and follow >> >> > the same approach from the Quickstarts, which means clone/build/run >> >> > Keycloak server from master. In this way, we can always it test against >> >> > the latest changes. >> >> > >> >> > Thoughts? >> >> > >> >> > >> >> > -- >> >> > >> >> > abstractj >> >> > _______________________________________________ >> >> > keycloak-dev mailing list >> >> > keycloak-dev at lists.jboss.org >> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > >> >> >> >> >> >> -- >> >> V?clav Muzik?? >> >> Quality Engineer >> >> Keycloak / Red Hat Single Sign-On >> >> Red Hat Czech s.r.o. >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> -- >> - abstractj >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev -- - abstractj From sthorger at redhat.com Thu Aug 16 09:39:34 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 16 Aug 2018 15:39:34 +0200 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: References: <20180813132345.GA28783@abstractj.org> Message-ID: On Thu, 16 Aug 2018 at 15:13, Bruno Oliveira wrote: > On Thu, Aug 16, 2018 at 5:34 AM Stian Thorgersen > wrote: > > > > How about this: > > > > * Rebuild Docker image for every PR. We can probably do this with a > webhook or somethin'. > > If we think about 3 repositories or more getting new PRs. This might > become a bit expensive, don't you think? I'd suggest another > alternative, but that would require a bot to trigger the docker build > on demand. This would be the flow proposed: > I didn't quite clarify what I meant. I only meant rebuild Docker image when a PR is merged into Keycloak, resulting in whenever master (keycloak/keycloak repo) changes the Keycloak images will be rebuilt. I believe we can quite easily setup a webhook in GitHub to trigger this rebuild in Docker Hub. Without the bot having to do anything. Docker Hub can however sometimes be a bit slow (up to an hour to schedule and build), so would be worth to allow tests to build the image locally if it's not updated. They can do this by checking out the Dockerfiles and running with build args to build from Keycloak master. > > 1. PR submitted and tested against the nightly build > 2. PR failed the author add a comment like @keycloak-bot > build-keycloak-server rerun > 3. The bot check if user has the rights for it, trigger the build of > the docker image and run the tests again > > This is just a suggestion which I understand, requires some time. But > this is the best which I can come up with. Or we can try what you > suggested and see how it goes. > > > * Write a script that can be shared to all projects that need a Keycloak > server. It would check the latest Docker hub image and see if it matches > master. If it doesn't it would build the image locally. Then it would start > the Keycloak server. Could use the same script for Node.js, Generic > Adapter, QuickStarts, etc.. > > +1 > > Once we reach an agreement on this, with whatever approach we decided. > We can file a Jira. > > > > > On Tue, 14 Aug 2018 at 15:10, Bruno Oliveira > wrote: > >> > >> Let's do this, I will change it to always build Keycloak from master > >> and later we can think about docker images. I think Docker images > >> would be better, at the same time, I'm not sure how fix the scenario > >> where feature A on Node.js adapter, depends on Feature B merged on > >> Keycloak master and both have to be merged/tested in the same day. > >> > >> Unless we do something like Hynek suggested. > >> On Tue, Aug 14, 2018 at 9:33 AM Hynek Mlnarik > wrote: > >> > > >> > Is nightly enough? > >> > > >> > Consider keycloak repo breaks due to some change and the quickstarts > cannot be built until this is fixed. In nightly, that would delay the > development to the next day. > >> > > >> > My vote is to either build Keycloak from master like Bruno suggested > or have a way documented to rebuild the "latest" image (regardless of > "nightly" name) anytime on demand to enable dependent changes to be > developed quickly. > >> > > >> > --Hynek > >> > > >> > On Tue, Aug 14, 2018 at 11:34 AM Vaclav Muzikar > wrote: > >> >> > >> >> +1 > >> >> > >> >> We've got a nightly CI job testing Node.js adapter against upstream > but > >> >> running it in Travis (with each PR) would make more sense. > >> >> > >> >> On Mon, Aug 13, 2018 at 3:40 PM Bruno Oliveira > wrote: > >> >> > >> >> > Good morning, > >> >> > > >> >> > Last week Pedro submitted a PR to the Node.js adapter, but the > build is > >> >> > failing because it depends on the changes from Keycloak server > master > >> >> > branch. > >> >> > > >> >> > Today we download the latest stable release from Keycloak to run > the > >> >> > integration tests for this adapter. I would like to change it and > follow > >> >> > the same approach from the Quickstarts, which means clone/build/run > >> >> > Keycloak server from master. In this way, we can always it test > against > >> >> > the latest changes. > >> >> > > >> >> > Thoughts? > >> >> > > >> >> > > >> >> > -- > >> >> > > >> >> > abstractj > >> >> > _______________________________________________ > >> >> > keycloak-dev mailing list > >> >> > keycloak-dev at lists.jboss.org > >> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> >> > > >> >> > >> >> > >> >> -- > >> >> V?clav Muzik?? > >> >> Quality Engineer > >> >> Keycloak / Red Hat Single Sign-On > >> >> Red Hat Czech s.r.o. > >> >> _______________________________________________ > >> >> keycloak-dev mailing list > >> >> keycloak-dev at lists.jboss.org > >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > >> > >> > >> -- > >> - abstractj > >> > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > - abstractj > From bruno at abstractj.org Thu Aug 16 11:21:51 2018 From: bruno at abstractj.org (Bruno Oliveira) Date: Thu, 16 Aug 2018 12:21:51 -0300 Subject: [keycloak-dev] Keycloak Node.js adapter integration tests on Travis In-Reply-To: References: <20180813132345.GA28783@abstractj.org> Message-ID: +1000 Created a Jira for that https://issues.jboss.org/browse/KEYCLOAK-8067 On Thu, Aug 16, 2018 at 10:39 AM Stian Thorgersen wrote: > > > > On Thu, 16 Aug 2018 at 15:13, Bruno Oliveira wrote: >> >> On Thu, Aug 16, 2018 at 5:34 AM Stian Thorgersen wrote: >> > >> > How about this: >> > >> > * Rebuild Docker image for every PR. We can probably do this with a webhook or somethin'. >> >> If we think about 3 repositories or more getting new PRs. This might >> become a bit expensive, don't you think? I'd suggest another >> alternative, but that would require a bot to trigger the docker build >> on demand. This would be the flow proposed: > > > I didn't quite clarify what I meant. I only meant rebuild Docker image when a PR is merged into Keycloak, resulting in whenever master (keycloak/keycloak repo) changes the Keycloak images will be rebuilt. > > I believe we can quite easily setup a webhook in GitHub to trigger this rebuild in Docker Hub. Without the bot having to do anything. > > Docker Hub can however sometimes be a bit slow (up to an hour to schedule and build), so would be worth to allow tests to build the image locally if it's not updated. They can do this by checking out the Dockerfiles and running with build args to build from Keycloak master. > >> >> >> 1. PR submitted and tested against the nightly build >> 2. PR failed the author add a comment like @keycloak-bot >> build-keycloak-server rerun >> 3. The bot check if user has the rights for it, trigger the build of >> the docker image and run the tests again >> >> This is just a suggestion which I understand, requires some time. But >> this is the best which I can come up with. Or we can try what you >> suggested and see how it goes. >> >> > * Write a script that can be shared to all projects that need a Keycloak server. It would check the latest Docker hub image and see if it matches master. If it doesn't it would build the image locally. Then it would start the Keycloak server. Could use the same script for Node.js, Generic Adapter, QuickStarts, etc.. >> >> +1 >> >> Once we reach an agreement on this, with whatever approach we decided. >> We can file a Jira. >> >> > >> > On Tue, 14 Aug 2018 at 15:10, Bruno Oliveira wrote: >> >> >> >> Let's do this, I will change it to always build Keycloak from master >> >> and later we can think about docker images. I think Docker images >> >> would be better, at the same time, I'm not sure how fix the scenario >> >> where feature A on Node.js adapter, depends on Feature B merged on >> >> Keycloak master and both have to be merged/tested in the same day. >> >> >> >> Unless we do something like Hynek suggested. >> >> On Tue, Aug 14, 2018 at 9:33 AM Hynek Mlnarik wrote: >> >> > >> >> > Is nightly enough? >> >> > >> >> > Consider keycloak repo breaks due to some change and the quickstarts cannot be built until this is fixed. In nightly, that would delay the development to the next day. >> >> > >> >> > My vote is to either build Keycloak from master like Bruno suggested or have a way documented to rebuild the "latest" image (regardless of "nightly" name) anytime on demand to enable dependent changes to be developed quickly. >> >> > >> >> > --Hynek >> >> > >> >> > On Tue, Aug 14, 2018 at 11:34 AM Vaclav Muzikar wrote: >> >> >> >> >> >> +1 >> >> >> >> >> >> We've got a nightly CI job testing Node.js adapter against upstream but >> >> >> running it in Travis (with each PR) would make more sense. >> >> >> >> >> >> On Mon, Aug 13, 2018 at 3:40 PM Bruno Oliveira wrote: >> >> >> >> >> >> > Good morning, >> >> >> > >> >> >> > Last week Pedro submitted a PR to the Node.js adapter, but the build is >> >> >> > failing because it depends on the changes from Keycloak server master >> >> >> > branch. >> >> >> > >> >> >> > Today we download the latest stable release from Keycloak to run the >> >> >> > integration tests for this adapter. I would like to change it and follow >> >> >> > the same approach from the Quickstarts, which means clone/build/run >> >> >> > Keycloak server from master. In this way, we can always it test against >> >> >> > the latest changes. >> >> >> > >> >> >> > Thoughts? >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > >> >> >> > abstractj >> >> >> > _______________________________________________ >> >> >> > keycloak-dev mailing list >> >> >> > keycloak-dev at lists.jboss.org >> >> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > >> >> >> >> >> >> >> >> >> -- >> >> >> V?clav Muzik?? >> >> >> Quality Engineer >> >> >> Keycloak / Red Hat Single Sign-On >> >> >> Red Hat Czech s.r.o. >> >> >> _______________________________________________ >> >> >> keycloak-dev mailing list >> >> >> keycloak-dev at lists.jboss.org >> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> >> >> >> -- >> >> - abstractj >> >> >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> -- >> - abstractj -- - abstractj From vmuzikar at redhat.com Thu Aug 16 12:50:50 2018 From: vmuzikar at redhat.com (Vaclav Muzikar) Date: Thu, 16 Aug 2018 18:50:50 +0200 Subject: [keycloak-dev] Default Locale Message-ID: Hi! Before I file a bug, I'd like to discuss one thing. How's the Default Locale (in realm's theme settings) supposed to work? It seems to have no effect at all because the realm settings is always overridden by "Accept-Language" HTTP header that's sent by browser. Shouldn't have the Default Locale higher priority than the HTTP header? Thanks! V. -- V?clav Muzik?? Quality Engineer Keycloak / Red Hat Single Sign-On Red Hat Czech s.r.o. From sthorger at redhat.com Fri Aug 17 07:26:07 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 17 Aug 2018 13:26:07 +0200 Subject: [keycloak-dev] Default Locale In-Reply-To: References: Message-ID: No, the accept-language locale should have a higher priority than the default locale. The realm default locale is only a fallback if the accept-language doesn't contain a locale Keycloak can provide or if for whatever unlikely reason one isn't sent. On Thu, 16 Aug 2018 at 19:59, Vaclav Muzikar wrote: > Hi! > Before I file a bug, I'd like to discuss one thing. > How's the Default Locale (in realm's theme settings) supposed to work? It > seems to have no effect at all because the realm settings is always > overridden by "Accept-Language" HTTP header that's sent by browser. > > Shouldn't have the Default Locale higher priority than the HTTP header? > > Thanks! > > V. > > -- > V?clav Muzik?? > Quality Engineer > Keycloak / Red Hat Single Sign-On > Red Hat Czech s.r.o. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From psilva at redhat.com Fri Aug 17 10:26:21 2018 From: psilva at redhat.com (Pedro Igor Silva) Date: Fri, 17 Aug 2018 11:26:21 -0300 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: <4dd6baaaf6654cb79441536c2118f798@elca.ch> References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Message-ID: On Tue, Aug 14, 2018 at 6:53 AM, Doswald Alistair wrote: > > * keycloak-authorization (https://github.com/cloudtrust > /keycloak-authorization): this module allows the use of the client > authorization system to prevent a user which is authenticated in a Keycloak > realm to access a given client. It works no matter which protocol is used, > and without the client having to support any extra protocol. Note: this > solution is a bit hacky, but necessary for one of our use-cases. > Regarding this extension. if I understood it correctly, it works like that. First from an admin perspective: 1) User creates a client an enable authorization services to it 2) User creates a resource "Keycloak Client Resource" and define any permission to it >From a server perspective: 1) Check if authorization services is enabled to the client to which the user is authenticating 2) Check whether or not permission to access the client is granted It seems that the main logic for this is at https://github.com/ cloudtrust/keycloak-authorization/blob/master/src/main/java/io/cloudtrust/ keycloak/protocol/LocalAuthorizationService.java. Is my understanding correct ? Would like to know if that is how it works today so we can start discussing alternatives. Regards. Pedro Igor From vmuzikar at redhat.com Fri Aug 17 10:27:15 2018 From: vmuzikar at redhat.com (Vaclav Muzikar) Date: Fri, 17 Aug 2018 16:27:15 +0200 Subject: [keycloak-dev] Default Locale In-Reply-To: References: Message-ID: Thanks for clarification. Is this behaviour documented somewhere? Docs [1] only briefly mention the Default Locale. [1] https://www.keycloak.org/docs/latest/server_admin/index.html#internationalization On Fri, Aug 17, 2018 at 1:26 PM Stian Thorgersen wrote: > No, the accept-language locale should have a higher priority than the > default locale. The realm default locale is only a fallback if the > accept-language doesn't contain a locale Keycloak can provide or if for > whatever unlikely reason one isn't sent. > > On Thu, 16 Aug 2018 at 19:59, Vaclav Muzikar wrote: > >> Hi! >> Before I file a bug, I'd like to discuss one thing. >> How's the Default Locale (in realm's theme settings) supposed to work? It >> seems to have no effect at all because the realm settings is always >> overridden by "Accept-Language" HTTP header that's sent by browser. >> >> Shouldn't have the Default Locale higher priority than the HTTP header? >> >> Thanks! >> >> V. >> >> -- >> V?clav Muzik?? >> Quality Engineer >> Keycloak / Red Hat Single Sign-On >> Red Hat Czech s.r.o. >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- V?clav Muzik?? Quality Engineer Keycloak / Red Hat Single Sign-On Red Hat Czech s.r.o. From sthorger at redhat.com Mon Aug 20 03:11:12 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 20 Aug 2018 09:11:12 +0200 Subject: [keycloak-dev] Default Locale In-Reply-To: References: Message-ID: No, it's not documented. Probably would make sense to do so feel free to create a JIRA. On Fri, 17 Aug 2018 at 16:27, Vaclav Muzikar wrote: > Thanks for clarification. Is this behaviour documented somewhere? Docs [1] > only briefly mention the Default Locale. > > [1] > https://www.keycloak.org/docs/latest/server_admin/index.html#internationalization > > On Fri, Aug 17, 2018 at 1:26 PM Stian Thorgersen > wrote: > >> No, the accept-language locale should have a higher priority than the >> default locale. The realm default locale is only a fallback if the >> accept-language doesn't contain a locale Keycloak can provide or if for >> whatever unlikely reason one isn't sent. >> >> On Thu, 16 Aug 2018 at 19:59, Vaclav Muzikar wrote: >> >>> Hi! >>> Before I file a bug, I'd like to discuss one thing. >>> How's the Default Locale (in realm's theme settings) supposed to work? It >>> seems to have no effect at all because the realm settings is always >>> overridden by "Accept-Language" HTTP header that's sent by browser. >>> >>> Shouldn't have the Default Locale higher priority than the HTTP header? >>> >>> Thanks! >>> >>> V. >>> >>> -- >>> V?clav Muzik?? >>> Quality Engineer >>> Keycloak / Red Hat Single Sign-On >>> Red Hat Czech s.r.o. >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > > -- > V?clav Muzik?? > Quality Engineer > Keycloak / Red Hat Single Sign-On > Red Hat Czech s.r.o. > From slaskawi at redhat.com Mon Aug 20 04:25:19 2018 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 20 Aug 2018 10:25:19 +0200 Subject: [keycloak-dev] Truststore SPI In-Reply-To: References: Message-ID: On Tue, Aug 14, 2018 at 11:30 PM Stian Thorgersen wrote: > Was thinking some more about how to share certificate functionality between > user authentication and client authentication. Potentially also to SAML. > > We could add a Truststore SPI. The TruststoreProvider interface would have > something like the following methods (quick mock stuff): > We already have a Truststore SPI [1]. If I remember correctly, it's being used when Keycloak needs to create an TLS connection to another IDP. Perhaps we could come up with another name, like CertificateSpi? [1] https://github.com/keycloak/keycloak/blob/08dbdeb57f678646a30a8ef44a76a8bf50f87e59/server-spi-private/src/main/java/org/keycloak/truststore/TruststoreSpi.java > > * boolean verifyCert(CertificateMatcher matcher) > * Set listExpiring() > This one should probably take a date as an argument (or maybe there should be an overloaded method that takes it?). > * void trustCert(Cert cert) > Probably addCert would be a better name since we have a removeTrustedCert method. > * void removeTrustedCert(String id) > * Cert getSavedCert(String id) > * void removeSavedCert(String id) > * void saveCert(Cert cert) > Plus, methods that list trusted and saved certs. > > We'd have a default built-in TruststoreProvider. Potentially delegating > some stuff to Elytron, but it would have to store certs in the db. It would > deal with checking if cert is still valid, call cert validation like the > user cert does, etc.. > That's a very good idea. Some of the interesting stuff is, like Certificates Revocation Lists are coming out of the box once we integrate Elytron [2]. [2] https://docs.jboss.org/author/display/WFLY/SSL+Configuration+using+Elytron+Subsystem#SSLConfigurationusingElytronSubsystem-UsingaCertificateRevocationList > > For user and client authentication we'd use verifyCert primarily. We'd have > something like UserCertMatcher.create()... and > ClientCertMatcher.create()... In the admin console we could add an option > to add trusted certs (not sure how that'd work in a cluster). It shouldn't be hard. We probably need a DB entity for that and let Infinispan do the caching. Although, I'm not sure if we have any constraints from the security perspective?? > For SAML > clients we'd delegate storing certs to the Truststore SPI instead of > storing the cert data alongside the client. The truststore would also add > some metadata in the db about certs so we can easily query for expiring > certs. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From vmuzikar at redhat.com Mon Aug 20 07:34:23 2018 From: vmuzikar at redhat.com (Vaclav Muzikar) Date: Mon, 20 Aug 2018 13:34:23 +0200 Subject: [keycloak-dev] Default Locale In-Reply-To: References: Message-ID: Created the JIRA: https://issues.jboss.org/browse/KEYCLOAK-8096 On Mon, Aug 20, 2018 at 9:11 AM Stian Thorgersen wrote: > No, it's not documented. Probably would make sense to do so feel free to > create a JIRA. > > On Fri, 17 Aug 2018 at 16:27, Vaclav Muzikar wrote: > >> Thanks for clarification. Is this behaviour documented somewhere? Docs >> [1] only briefly mention the Default Locale. >> >> [1] >> https://www.keycloak.org/docs/latest/server_admin/index.html#internationalization >> >> On Fri, Aug 17, 2018 at 1:26 PM Stian Thorgersen >> wrote: >> >>> No, the accept-language locale should have a higher priority than the >>> default locale. The realm default locale is only a fallback if the >>> accept-language doesn't contain a locale Keycloak can provide or if for >>> whatever unlikely reason one isn't sent. >>> >>> On Thu, 16 Aug 2018 at 19:59, Vaclav Muzikar >>> wrote: >>> >>>> Hi! >>>> Before I file a bug, I'd like to discuss one thing. >>>> How's the Default Locale (in realm's theme settings) supposed to work? >>>> It >>>> seems to have no effect at all because the realm settings is always >>>> overridden by "Accept-Language" HTTP header that's sent by browser. >>>> >>>> Shouldn't have the Default Locale higher priority than the HTTP header? >>>> >>>> Thanks! >>>> >>>> V. >>>> >>>> -- >>>> V?clav Muzik?? >>>> Quality Engineer >>>> Keycloak / Red Hat Single Sign-On >>>> Red Hat Czech s.r.o. >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >> >> -- >> V?clav Muzik?? >> Quality Engineer >> Keycloak / Red Hat Single Sign-On >> Red Hat Czech s.r.o. >> > -- V?clav Muzik?? Quality Engineer Keycloak / Red Hat Single Sign-On Red Hat Czech s.r.o. From alistair.doswald at elca.ch Mon Aug 20 07:36:19 2018 From: alistair.doswald at elca.ch (Doswald Alistair) Date: Mon, 20 Aug 2018 11:36:19 +0000 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Message-ID: <806a7fe7a66c4f88afb54455c8edf0a0@elca.ch> Hello, I?ve now received two questions concerning how the authorization module works, so I thought best to answer them both at the same time. Pedro Igor?s description (attached below) is broadly correct, but I?ll elaborate more on the principals. As mentioned, it?s a bit hacky. The solutions basically hijacks the existing authorization system used for OIDC clients and UMA, but makes it available to other client types as well. The initial step is allowing all clients to define authorization rule by giving them access to the authorization GUI. This allows us to use all the policy and permissions? systems to create rules for the user access. The one thing that must be done is to create the "Keycloak Client Resource" (and indeed define any permission to it) in order to activate the local evaluation for the client. Using a specifically named resource allows us to separate the case from an OIDC client that also wants to use the regular authorization services. The limitation when compared to UMA is that you can?t protect a specific resource, it?s all the client or nothing. When a user attempts to connect with one of those clients, we let the authentication process continue until the authentication process for keycloak is finished (if it is not already established), but check the authorization status before an actual token is sent to that client. The authorization process (in class LocalAuthorizationService.java) also uses existing systems. We use the current state of the system to construct an AccessToken, which we then submit to an AuthorizationProvider. If the user is permitted to continue, he will receive the token. If not, he will receive a 403 FORBIDDEN. Without the token for a specific client, he should never be able to access that client. But whether the user is authorized for a client or not, the user is authenticated in keycloak, and can continue to enjoy SSO services for any client for which he is authorized. Of course, this is all only valid for keycloak 3.4.3.Final. I haven?t yet gotten around to checking the changes in keycloak 4.X, and seeing if the system is still valid with the changes to the authorization system. I?ll also readily admit that our current solution lacks polish (and there?s probably some edge cases that we haven?t thought of). @Stian: for the suggestion of roles, I don?t think it would be much different from our solution while using a role policy, so I could definitely see it working. However, as far as I can see, using policies and permissions allows for more flexibility. By extending the system to client scopes, do you mean allowing a more fine grained control on what information is transmitted for a given user depending on what permissions he has? Also @Stian: * For keycloak wsfed, we?ll continue updating it as long as we have a use case for it, no problems. * I?ve started on the work necessary to do a pull request for the script mapper. From what I can see, I only need to do some refactoring to use existing methods that would simplify the code, and then I need to see how the other mappers are tested so that I can provide some tests. Someone also told me I need to submit some documentation, so I?ll look into that as well * For the keycloak-export module, I need to discuss it some more with a couple of people. This should happen tomorrow, so I?ll get back to you once I can give you a clear answer. From: Pedro Igor Silva Sent: vendredi 17 ao?t 2018 16:26 To: Doswald Alistair Cc: keycloak-dev at lists.jboss.org Subject: Re: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project On Tue, Aug 14, 2018 at 6:53 AM, Doswald Alistair > wrote: * keycloak-authorization (https://github.com/cloudtrust/keycloak-authorization): this module allows the use of the client authorization system to prevent a user which is authenticated in a Keycloak realm to access a given client. It works no matter which protocol is used, and without the client having to support any extra protocol. Note: this solution is a bit hacky, but necessary for one of our use-cases. Regarding this extension. if I understood it correctly, it works like that. First from an admin perspective: 1) User creates a client an enable authorization services to it 2) User creates a resource "Keycloak Client Resource" and define any permission to it From a server perspective: 1) Check if authorization services is enabled to the client to which the user is authenticating 2) Check whether or not permission to access the client is granted It seems that the main logic for this is at https://github.com/cloudtrust/keycloak-authorization/blob/master/src/main/java/io/cloudtrust/keycloak/protocol/LocalAuthorizationService.java. Is my understanding correct ? Would like to know if that is how it works today so we can start discussing alternatives. Regards. Pedro Igor From: Stian Thorgersen Sent: jeudi 16 ao?t 2018 10:30 To: Doswald Alistair Cc: keycloak-dev Subject: Re: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project On Tue, 14 Aug 2018 at 12:03, Doswald Alistair > wrote: Hello, I just wanted to let this mailing list know that for the Cloudtrust project (https://github.com/cloudtrust), we have developed a certain number modules for Keycloak. These are currently compatible with the version 3.4.3.Final of Keycloak, but we will make them compatible with Keycloak 4.X (where X will be the latest sub-version of Keycloak when we start working on this) as soon as we can. These modules are: * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an implementation of the WS-Federation protocol for keycloak. This allows to select the WS-Federation protocol for Keycloak clients and for identity brokers. This is great and pleased that you are maintaining this. We simply don't have the capacity to have this incorporated into Keycloak as we lack expertise on WS-Fed and have to many other things to maintain. * keycloak-authorization (https://github.com/cloudtrust/keycloak-authorization): this module allows the use of the client authorization system to prevent a user which is authenticated in a Keycloak realm to access a given client. It works no matter which protocol is used, and without the client having to support any extra protocol. Note: this solution is a bit hacky, but necessary for one of our use-cases. Sounds interesting. Can you write some more details about this? In the past I've considered if we should have something like required roles to use to a client. That would by not allowing a user to authenticate with a client, but also could prevent adding audience for a service the user isn't allowed to access. Could perhaps take it one step further and prevent client scopes that users are not allowed to access. * keycloak-client-mappers (https://github.com/cloudtrust/keycloak-client-mappers): a module for adding any mappers that we might need that are not yet part of Keycloak. Currently only contains a JavaScript mapper for SAML, analogous to the OIDC script mapper. I've noticed that there's an open issue for this feature (https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could submit this code not as a module but a solution to the issue. SAML JavaScript protocol mapper would be great. If you submit with tests and docs we'd love to have this in. * keycloak-export (https://github.com/cloudtrust/keycloak-export): a module adding an endpoint to fully export a realm while Keycloak is still running (no need for restarts!). Can you write some more details about this? In the past we've considered this, but not added it mainly for a few reasons, including: * What happens if there are changes while doing the export, we where thinking we'd have to pause the server from receiving requests during this export * We don't want a REST endpoint that can return credentials and other secrets. Then there's also the size of the export. It would have to store the export on disc, but then what about clustering. One potential would be to develop a tool that can be executed from anywhere that has access to the db. It would use Keycloak code base, but instead of starting a server it would allow things like running an export from the db, migrate the db, etc. Cheers, Alistair PS: I mailed this to both dev and user mailing lists as I believe it may interest members of both mailing lists. However, upon sending to the dev mailing list the first time it bounced. This is the second attempt. _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From hmlnarik at redhat.com Mon Aug 20 09:38:07 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Mon, 20 Aug 2018 15:38:07 +0200 Subject: [keycloak-dev] [keycloak-user] Keycloak Proxy Rename In-Reply-To: <20180625154229.GA17717@abstractj.org> References: <20180625154229.GA17717@abstractj.org> Message-ID: Based on discussion on Stian, let me reopen this topic and add a suggestion. How about "Standalone Keycloak Adapter" or just "Standalone Adapter"? On Mon, Jun 25, 2018 at 5:44 PM Bruno Oliveira wrote: > Good afternoon, > > We are considering to transfer or fork the keycloak-proxy[1] to Keycloak > organization. In order to accomplish that, I've been working with Rohith > updating some of its dependencies[2]. > > While discussing with our team, we reached the conclusion that call it a > proxy could potentially increase the scope of the project and also give > people the wrong idea. Because would be expected things like load > balancing, > rate limiting, and other features. That's not what we want right now. > > I would like to gather some feedback from the community before we move > forward. > So please vote on the following Doodle: > > https://doodle.com/poll/gux626ktscgpr96t > > Also, feel free to suggest other names and it will be included. > > [1] - https://github.com/gambol99/keycloak-proxy > [2] - https://issues.jboss.org/browse/KEYCLOAK-7265 > > > -- > > abstractj > _______________________________________________ > keycloak-user mailing list > keycloak-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-user > From psilva at redhat.com Mon Aug 20 11:40:56 2018 From: psilva at redhat.com (Pedro Igor Silva) Date: Mon, 20 Aug 2018 12:40:56 -0300 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: <806a7fe7a66c4f88afb54455c8edf0a0@elca.ch> References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> <806a7fe7a66c4f88afb54455c8edf0a0@elca.ch> Message-ID: On Mon, Aug 20, 2018 at 8:36 AM, Doswald Alistair wrote: > Hello, > > > > I?ve now received two questions concerning how the authorization module > works, so I thought best to answer them both at the same time. > > > > Pedro Igor?s description (attached below) is broadly correct, but I?ll > elaborate more on the principals. As mentioned, it?s a bit hacky. The > solutions basically hijacks the existing authorization system used for OIDC > clients and UMA, but makes it available to other client types as well. > > > > The initial step is allowing all clients to define authorization rule by > giving them access to the authorization GUI. This allows us to use all the > policy and permissions? systems to create rules for the user access. The > one thing that must be done is to create the "Keycloak Client Resource" > (and indeed define any permission to it) in order to activate the local > evaluation for the client. Using a specifically named resource allows us to > separate the case from an OIDC client that also wants to use the regular > authorization services. The limitation when compared to UMA is that you > can?t protect a specific resource, it?s all the client or nothing. > > > > When a user attempts to connect with one of those clients, we let the > authentication process continue until the authentication process for > keycloak is finished (if it is not already established), but check the > authorization status before an actual token is sent to that client. The > authorization process (in class LocalAuthorizationService.java) also uses > existing systems. We use the current state of the system to construct an > AccessToken, which we then submit to an AuthorizationProvider. If the user > is permitted to continue, he will receive the token. If not, he will > receive a 403 FORBIDDEN. Without the token for a specific client, he should > never be able to access that client. But whether the user is authorized for > a client or not, the user is authenticated in keycloak, and can continue to > enjoy SSO services for any client for which he is authorized. > > > > Of course, this is all only valid for keycloak 3.4.3.Final. I haven?t yet > gotten around to checking the changes in keycloak 4.X, and seeing if the > system is still valid with the changes to the authorization system. I?ll > also readily admit that our current solution lacks polish (and there?s > probably some edge cases that we haven?t thought of). > Not sure if you are aware of the admin fine-grained permission functionality, which you can use today to restrict what users can do in the admin console. I think we could leverage this existing functionality by including a specific scope. A scope like "login" would be enough, I think, so you could just associate any policy you want to this scope in order to restrict who/how/when is allowed to authenticate with a client. > > > @Stian: for the suggestion of roles, I don?t think it would be much > different from our solution while using a role policy, so I could > definitely see it working. However, as far as I can see, using policies and > permissions allows for more flexibility. By extending the system to client > scopes, do you mean allowing a more fine grained control on what > information is transmitted for a given user depending on what permissions > he has? > > > > Also @Stian: > > - For keycloak wsfed, we?ll continue updating it as long as we have a > use case for it, no problems. > - I?ve started on the work necessary to do a pull request for the > script mapper. From what I can see, I only need to do some refactoring to > use existing methods that would simplify the code, and then I need to see > how the other mappers are tested so that I can provide some tests. Someone > also told me I need to submit some documentation, so I?ll look into that as > well > - For the keycloak-export module, I need to discuss it some more with > a couple of people. This should happen tomorrow, so I?ll get back to you > once I can give you a clear answer. > > > > *From:* Pedro Igor Silva > *Sent:* vendredi 17 ao?t 2018 16:26 > *To:* Doswald Alistair > *Cc:* keycloak-dev at lists.jboss.org > *Subject:* Re: [keycloak-dev] Keycloak Modules developed for the > Cloudtrust project > > > > On Tue, Aug 14, 2018 at 6:53 AM, Doswald Alistair < > alistair.doswald at elca.ch> wrote: > > > * keycloak-authorization (https://github.com/cloudtrust/keycloak- > authorization): this module allows the use of the client authorization > system to prevent a user which is authenticated in a Keycloak realm to > access a given client. It works no matter which protocol is used, and > without the client having to support any extra protocol. Note: this > solution is a bit hacky, but necessary for one of our use-cases. > > > > Regarding this extension. if I understood it correctly, it works like > that. First from an admin perspective: > > > > 1) User creates a client an enable authorization services to it > > 2) User creates a resource "Keycloak Client Resource" and define any > permission to it > > > > From a server perspective: > > > > 1) Check if authorization services is enabled to the client to which the > user is authenticating > > 2) Check whether or not permission to access the client is granted > > > > It seems that the main logic for this is at https://github.com/ > cloudtrust/keycloak-authorization/blob/master/src/main/java/io/cloudtrust/ > keycloak/protocol/LocalAuthorizationService.java. > > > > Is my understanding correct ? Would like to know if that is how it works > today so we can start discussing alternatives. > > > > Regards. > > Pedro Igor > > > > > > *From:* Stian Thorgersen > *Sent:* jeudi 16 ao?t 2018 10:30 > *To:* Doswald Alistair > *Cc:* keycloak-dev > *Subject:* Re: [keycloak-dev] Keycloak Modules developed for the > Cloudtrust project > > > > > > On Tue, 14 Aug 2018 at 12:03, Doswald Alistair > wrote: > > Hello, > > I just wanted to let this mailing list know that for the Cloudtrust > project (https://github.com/cloudtrust), we have developed a certain > number modules for Keycloak. These are currently compatible with the > version 3.4.3.Final of Keycloak, but we will make them compatible with > Keycloak 4.X (where X will be the latest sub-version of Keycloak when we > start working on this) as soon as we can. These modules are: > > * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an > implementation of the WS-Federation protocol for keycloak. This allows to > select the WS-Federation protocol for Keycloak clients and for identity > brokers. > > > > This is great and pleased that you are maintaining this. We simply don't > have the capacity to have this incorporated into Keycloak as we lack > expertise on WS-Fed and have to many other things to maintain. > > > > > * keycloak-authorization (https://github.com/cloudtrust/keycloak- > authorization): this module allows the use of the client authorization > system to prevent a user which is authenticated in a Keycloak realm to > access a given client. It works no matter which protocol is used, and > without the client having to support any extra protocol. Note: this > solution is a bit hacky, but necessary for one of our use-cases. > > > > Sounds interesting. Can you write some more details about this? In the > past I've considered if we should have something like required roles to use > to a client. That would by not allowing a user to authenticate with a > client, but also could prevent adding audience for a service the user isn't > allowed to access. Could perhaps take it one step further and prevent > client scopes that users are not allowed to access. > > > > > * keycloak-client-mappers (https://github.com/cloudtrust/keycloak-client- > mappers): a module for adding any mappers that we might need that are not > yet part of Keycloak. Currently only contains a JavaScript mapper for SAML, > analogous to the OIDC script mapper. I've noticed that there's an open > issue for this feature (https://issues.jboss.org/browse/KEYCLOAK-5520). > If desirable I could submit this code not as a module but a solution to the > issue. > > > > SAML JavaScript protocol mapper would be great. If you submit with tests > and docs we'd love to have this in. > > > > > * keycloak-export (https://github.com/cloudtrust/keycloak-export): a > module adding an endpoint to fully export a realm while Keycloak is still > running (no need for restarts!). > > > > Can you write some more details about this? In the past we've considered > this, but not added it mainly for a few reasons, including: > > > > * What happens if there are changes while doing the export, we where > thinking we'd have to pause the server from receiving requests during this > export > > * We don't want a REST endpoint that can return credentials and other > secrets. Then there's also the size of the export. It would have to store > the export on disc, but then what about clustering. One potential would be > to develop a tool that can be executed from anywhere that has access to the > db. It would use Keycloak code base, but instead of starting a server it > would allow things like running an export from the db, migrate the db, etc. > > > > > Cheers, > > Alistair > > PS: I mailed this to both dev and user mailing lists as I believe it may > interest members of both mailing lists. However, upon sending to the dev > mailing list the first time it bounced. This is the second attempt. > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > From bruno at abstractj.org Mon Aug 20 11:54:42 2018 From: bruno at abstractj.org (Bruno Oliveira) Date: Mon, 20 Aug 2018 12:54:42 -0300 Subject: [keycloak-dev] [keycloak-user] Keycloak Proxy Rename In-Reply-To: References: <20180625154229.GA17717@abstractj.org> Message-ID: Only to give a brief context for people not aware of it. Keycloak Generic Adapter was not well accepted, because the naming is too vague. So we have to reopen this discussion and think about a better naming. During our team call today I suggested just "keycloak-adapter", which would cover the apps which don't have its own specific adapter solution. That said, maybe we should open a new poll? I just created a new one where people can vote/suggest: https://poll.ly/#/Lbww4ebG On Mon, Aug 20, 2018 at 10:38 AM Hynek Mlnarik wrote: > > Based on discussion on Stian, let me reopen this topic and add a suggestion. > > How about "Standalone Keycloak Adapter" or just "Standalone Adapter"? > > > On Mon, Jun 25, 2018 at 5:44 PM Bruno Oliveira wrote: >> >> Good afternoon, >> >> We are considering to transfer or fork the keycloak-proxy[1] to Keycloak >> organization. In order to accomplish that, I've been working with Rohith >> updating some of its dependencies[2]. >> >> While discussing with our team, we reached the conclusion that call it a >> proxy could potentially increase the scope of the project and also give >> people the wrong idea. Because would be expected things like load balancing, >> rate limiting, and other features. That's not what we want right now. >> >> I would like to gather some feedback from the community before we move forward. >> So please vote on the following Doodle: >> >> https://doodle.com/poll/gux626ktscgpr96t >> >> Also, feel free to suggest other names and it will be included. >> >> [1] - https://github.com/gambol99/keycloak-proxy >> [2] - https://issues.jboss.org/browse/KEYCLOAK-7265 >> >> >> -- >> >> abstractj >> _______________________________________________ >> keycloak-user mailing list >> keycloak-user at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-user -- - abstractj From thomas.darimont at googlemail.com Mon Aug 20 14:05:44 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Mon, 20 Aug 2018 20:05:44 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final Message-ID: Hello Keycloak team, has anyone encountered some performance issues after upgrading 3.4.3 to 4.x (4.3.0)? Today I noticed a performance regression while preparing an upgrade from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. In our test environment, we have around ~100k test users stored in a postgres-backed database. When we started the server with the new Keycloak version, the migration went through, and everything looked fine at first, but when we tried to browse the list of users via the admin-console, we noticed that the CPU and memory consumption of the server increased significantly, up to a point where Keycloak crashed with an OOME. All previous Keycloak versions including 3.4.3 were very modest with their memory requirements and quite happy with ~1g heap. However, that seems to have changed in Keycloak 4.3.0 - there we needed at least 4g to prevent Keycloak from crashing with an OOME. Furthermore, we noticed that the response times for browsing the paginated user view increased significantly as well: In Keycloak 3.4.3 the average time to load a user page is ~80ms. In Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation takes ~7 seconds for a test realm with just 10k users. In the test realm with 100k users, the time to load a single page in the users listing was 66 seconds for version 4.3.0, on average - compared to quite stable 80ms in 3.4.3. The database query that is executed by Keycloak 4.3.0 runs in ~1.5 seconds for 100k users, so I assume the processing logic in Keycloak is the culprit. The problem of long load-times can be reproduced with the Keycloak docker images and the in-memory database. I also created a small example project that creates some users with just a few attributes in a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres database to reproduce the problem. https://github.com/thomasdarimont/kc-user-regression-tester Cheers, Thomas From aszczucz at redhat.com Mon Aug 20 17:04:18 2018 From: aszczucz at redhat.com (Alex Szczuczko) Date: Mon, 20 Aug 2018 15:04:18 -0600 Subject: [keycloak-dev] [keycloak-user] Keycloak Proxy Rename In-Reply-To: References: <20180625154229.GA17717@abstractj.org> Message-ID: <153479905785.6959.1361851594171681280@tyrfing> In thinking a new name, I tried to look hard at these things: 1. what this software actually does. 2. what makes this software desirable to a user. 3. what "adapter" has meant for keycloak in the past. I'm not the best person to answer these questions, but here's what I've dug up: 1. Accepts HTTP requests and talks with Keycloak via OIDC to see if the client it serves should treat the requests as authenticated and/or authorized. 2. It avoids the need to install a bit of Keycloak software into the users' applications. 3. According to the docs[1]: Keycloak client adapters are libraries that makes it very easy to secure applications and services with Keycloak ... our adapters easy to use and they require less boilerplate code than what is typically required by a library. #1 is what we've been focusing on with names like "proxy". The reasons such names are dissatisfying is there is nothing unique about sitting in between two endpoints and doing stuff. So, we need to look at what that "stuff" means for Keycloak. #3 in combination with #2 tells us what this "stuff" means for Keycloak. This new software is clearly not an adapter. Actually, this new software accomplishes the mission of an adapter better than adapters themselves! Following that logic, Superadapter is my main proposal for a new name. Maybe throw in OIDC (oidc-superadapter) if there's ever going to be a saml-superadapter. Alternatively, we could focus on the lack of an adapter, with names based on terms like Adapterless: - AKI: Adapterless Keycloak Integrator - KOSA: Keycloak OIDC Sans-Adapter - AKOS: Adapterless Keycloak OIDC Server - KOAF: Keycloak OIDC Adapter-Free - etc... Alex [1] https://www.keycloak.org/docs/latest/securing_apps/index.html#what-are-client-adapters Quoting Bruno Oliveira (2018-08-20 09:54:42) > Only to give a brief context for people not aware of it. Keycloak > Generic Adapter was not well accepted, because the naming is too > vague. So we have to reopen this discussion and think about a better > naming. > > During our team call today I suggested just "keycloak-adapter", which > would cover the apps which don't have its own specific adapter > solution. > > That said, maybe we should open a new poll? I just created a new one > where people can vote/suggest: > > https://poll.ly/#/Lbww4ebG From max.catarino at rps.com.br Tue Aug 21 00:00:33 2018 From: max.catarino at rps.com.br (max.catarino at rps.com.br) Date: Tue, 21 Aug 2018 01:00:33 -0300 Subject: [keycloak-dev] It's possible to add a totp using keycloak-admin-client 4.3.0? Message-ID: Hello. It's possible to add a totp using keycloak-admin-client 4.3.0? There is an example anyware? Best regards. From alistair.doswald at elca.ch Tue Aug 21 04:44:05 2018 From: alistair.doswald at elca.ch (Doswald Alistair) Date: Tue, 21 Aug 2018 08:44:05 +0000 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> <806a7fe7a66c4f88afb54455c8edf0a0@elca.ch> Message-ID: <813d996081ae4fcd83fce37474dd7acb@elca.ch> Actually, I am aware of the fine-grained permission functionality, but maybe not when I was asked to provide authorization for access to a client. Whatever the reason, I hadn?t considered using it before, but it is quite a good fit: it would be a client specific restriction for which policies are used. Of course, in reality, it is more or less the same as the cloudtrust authorization module, except that it?s resources in the realm management authorization which are used to represent the clients and a new ?login? scope which would be used to evaluate the login function. It seems less hacky then my solution, as there?s already a GUI function to create the resource for a given client, and no appropriation of a different functionality . After that I could even reuse my existing code, it would be a matter of getting the correct resource and scope, and the rest of the logic remains unchanged. I?ll look into it when I migrate the authorization module to keycloak 4.X. Of course, if everyone feels that this functionality (or similar) would be better implemented directly into keycloak, I?d be even happier. Cheers, Alistair From: Pedro Igor Silva Sent: lundi 20 ao?t 2018 17:41 To: Doswald Alistair Cc: stian at redhat.com; keycloak-dev Subject: Re: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project On Mon, Aug 20, 2018 at 8:36 AM, Doswald Alistair > wrote: Hello, I?ve now received two questions concerning how the authorization module works, so I thought best to answer them both at the same time. Pedro Igor?s description (attached below) is broadly correct, but I?ll elaborate more on the principals. As mentioned, it?s a bit hacky. The solutions basically hijacks the existing authorization system used for OIDC clients and UMA, but makes it available to other client types as well. The initial step is allowing all clients to define authorization rule by giving them access to the authorization GUI. This allows us to use all the policy and permissions? systems to create rules for the user access. The one thing that must be done is to create the "Keycloak Client Resource" (and indeed define any permission to it) in order to activate the local evaluation for the client. Using a specifically named resource allows us to separate the case from an OIDC client that also wants to use the regular authorization services. The limitation when compared to UMA is that you can?t protect a specific resource, it?s all the client or nothing. When a user attempts to connect with one of those clients, we let the authentication process continue until the authentication process for keycloak is finished (if it is not already established), but check the authorization status before an actual token is sent to that client. The authorization process (in class LocalAuthorizationService.java) also uses existing systems. We use the current state of the system to construct an AccessToken, which we then submit to an AuthorizationProvider. If the user is permitted to continue, he will receive the token. If not, he will receive a 403 FORBIDDEN. Without the token for a specific client, he should never be able to access that client. But whether the user is authorized for a client or not, the user is authenticated in keycloak, and can continue to enjoy SSO services for any client for which he is authorized. Of course, this is all only valid for keycloak 3.4.3.Final. I haven?t yet gotten around to checking the changes in keycloak 4.X, and seeing if the system is still valid with the changes to the authorization system. I?ll also readily admit that our current solution lacks polish (and there?s probably some edge cases that we haven?t thought of). Not sure if you are aware of the admin fine-grained permission functionality, which you can use today to restrict what users can do in the admin console. I think we could leverage this existing functionality by including a specific scope. A scope like "login" would be enough, I think, so you could just associate any policy you want to this scope in order to restrict who/how/when is allowed to authenticate with a client. @Stian: for the suggestion of roles, I don?t think it would be much different from our solution while using a role policy, so I could definitely see it working. However, as far as I can see, using policies and permissions allows for more flexibility. By extending the system to client scopes, do you mean allowing a more fine grained control on what information is transmitted for a given user depending on what permissions he has? Also @Stian: * For keycloak wsfed, we?ll continue updating it as long as we have a use case for it, no problems. * I?ve started on the work necessary to do a pull request for the script mapper. From what I can see, I only need to do some refactoring to use existing methods that would simplify the code, and then I need to see how the other mappers are tested so that I can provide some tests. Someone also told me I need to submit some documentation, so I?ll look into that as well * For the keycloak-export module, I need to discuss it some more with a couple of people. This should happen tomorrow, so I?ll get back to you once I can give you a clear answer. From: Pedro Igor Silva > Sent: vendredi 17 ao?t 2018 16:26 To: Doswald Alistair > Cc: keycloak-dev at lists.jboss.org Subject: Re: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project On Tue, Aug 14, 2018 at 6:53 AM, Doswald Alistair > wrote: * keycloak-authorization (https://github.com/cloudtrust/keycloak-authorization): this module allows the use of the client authorization system to prevent a user which is authenticated in a Keycloak realm to access a given client. It works no matter which protocol is used, and without the client having to support any extra protocol. Note: this solution is a bit hacky, but necessary for one of our use-cases. Regarding this extension. if I understood it correctly, it works like that. First from an admin perspective: 1) User creates a client an enable authorization services to it 2) User creates a resource "Keycloak Client Resource" and define any permission to it From a server perspective: 1) Check if authorization services is enabled to the client to which the user is authenticating 2) Check whether or not permission to access the client is granted It seems that the main logic for this is at https://github.com/cloudtrust/keycloak-authorization/blob/master/src/main/java/io/cloudtrust/keycloak/protocol/LocalAuthorizationService.java. Is my understanding correct ? Would like to know if that is how it works today so we can start discussing alternatives. Regards. Pedro Igor From: Stian Thorgersen > Sent: jeudi 16 ao?t 2018 10:30 To: Doswald Alistair > Cc: keycloak-dev > Subject: Re: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project On Tue, 14 Aug 2018 at 12:03, Doswald Alistair > wrote: Hello, I just wanted to let this mailing list know that for the Cloudtrust project (https://github.com/cloudtrust), we have developed a certain number modules for Keycloak. These are currently compatible with the version 3.4.3.Final of Keycloak, but we will make them compatible with Keycloak 4.X (where X will be the latest sub-version of Keycloak when we start working on this) as soon as we can. These modules are: * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an implementation of the WS-Federation protocol for keycloak. This allows to select the WS-Federation protocol for Keycloak clients and for identity brokers. This is great and pleased that you are maintaining this. We simply don't have the capacity to have this incorporated into Keycloak as we lack expertise on WS-Fed and have to many other things to maintain. * keycloak-authorization (https://github.com/cloudtrust/keycloak-authorization): this module allows the use of the client authorization system to prevent a user which is authenticated in a Keycloak realm to access a given client. It works no matter which protocol is used, and without the client having to support any extra protocol. Note: this solution is a bit hacky, but necessary for one of our use-cases. Sounds interesting. Can you write some more details about this? In the past I've considered if we should have something like required roles to use to a client. That would by not allowing a user to authenticate with a client, but also could prevent adding audience for a service the user isn't allowed to access. Could perhaps take it one step further and prevent client scopes that users are not allowed to access. * keycloak-client-mappers (https://github.com/cloudtrust/keycloak-client-mappers): a module for adding any mappers that we might need that are not yet part of Keycloak. Currently only contains a JavaScript mapper for SAML, analogous to the OIDC script mapper. I've noticed that there's an open issue for this feature (https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could submit this code not as a module but a solution to the issue. SAML JavaScript protocol mapper would be great. If you submit with tests and docs we'd love to have this in. * keycloak-export (https://github.com/cloudtrust/keycloak-export): a module adding an endpoint to fully export a realm while Keycloak is still running (no need for restarts!). Can you write some more details about this? In the past we've considered this, but not added it mainly for a few reasons, including: * What happens if there are changes while doing the export, we where thinking we'd have to pause the server from receiving requests during this export * We don't want a REST endpoint that can return credentials and other secrets. Then there's also the size of the export. It would have to store the export on disc, but then what about clustering. One potential would be to develop a tool that can be executed from anywhere that has access to the db. It would use Keycloak code base, but instead of starting a server it would allow things like running an export from the db, migrate the db, etc. Cheers, Alistair PS: I mailed this to both dev and user mailing lists as I believe it may interest members of both mailing lists. However, upon sending to the dev mailing list the first time it bounced. This is the second attempt. _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From sthorger at redhat.com Tue Aug 21 06:58:21 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 21 Aug 2018 12:58:21 +0200 Subject: [keycloak-dev] It's possible to add a totp using keycloak-admin-client 4.3.0? In-Reply-To: References: Message-ID: Not sure that makes all that much sense. How are you using the admin client? On Tue, 21 Aug 2018 at 07:03, wrote: > > > Hello. > > It's possible to add a totp using keycloak-admin-client 4.3.0? > There is an example anyware? > > Best regards. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Tue Aug 21 14:13:26 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 21 Aug 2018 20:13:26 +0200 Subject: [keycloak-dev] [keycloak-user] Keycloak Proxy Rename In-Reply-To: <1A3C52DFCD06494D8528644858247BF01C180537@EX10MBOX03.pnnl.gov> References: <20180625154229.GA17717@abstractj.org> <153479905785.6959.1361851594171681280@tyrfing> <1A3C52DFCD06494D8528644858247BF01C180537@EX10MBOX03.pnnl.gov> Message-ID: Sure, proxy is the obvious name, but for reasons already mentioned by Bruno it's not really an option for us. It comes from the Keycloak team, so it should have the Keycloak name in it. I agree that doesn't automatically state it's a generic OIDC adapter, but I'd like to keep our name in there. >From the suggestions so far there are two I like: * Keycloak Gatekeeper - suggested by Thomas on the poll. I really like this and it fits nicely with Keycloak. It's also so much cooler than proxy/standalone/etc. * Keycloak Standalone Adapter On Tue, 21 Aug 2018 at 04:27, Fox, Kevin M wrote: > Coming from the outside world, I mostly searched for oidc and proxy as > thats what I needed. I found oauth2_proxy after a little searching, but > have been disappointed in how slow its releases are. Bugs aren't getting > fixed quickly. When I looked at keycloak-proxy initially, I didn't look > closer for a while as i thought is was keycloak specific. > > So, something like oidc-proxy might get you more successful hits. > > Thanks, > Kevin > ________________________________________ > From: keycloak-user-bounces at lists.jboss.org [ > keycloak-user-bounces at lists.jboss.org] on behalf of Alex Szczuczko [ > aszczucz at redhat.com] > Sent: Monday, August 20, 2018 2:04 PM > To: Bruno Oliveira; Hynek Mlnarik > Cc: keycloak-dev; keycloak-user > Subject: Re: [keycloak-user] Keycloak Proxy Rename > > In thinking a new name, I tried to look hard at these things: > > 1. what this software actually does. > > 2. what makes this software desirable to a user. > > 3. what "adapter" has meant for keycloak in the past. > > I'm not the best person to answer these questions, but here's what I've > dug up: > > 1. Accepts HTTP requests and talks with Keycloak via OIDC to see if > the client it serves should treat the requests as authenticated > and/or authorized. > > 2. It avoids the need to install a bit of Keycloak software into the > users' applications. > > 3. According to the docs[1]: Keycloak client adapters are libraries > that makes it very easy to secure applications and services with > Keycloak ... our adapters easy to use and they require less > boilerplate code than what is typically required by a library. > > #1 is what we've been focusing on with names like "proxy". The reasons > such names are dissatisfying is there is nothing unique about sitting in > between two endpoints and doing stuff. So, we need to look at what that > "stuff" means for Keycloak. > > #3 in combination with #2 tells us what this "stuff" means for Keycloak. > This new software is clearly not an adapter. Actually, this new software > accomplishes the mission of an adapter better than adapters themselves! > > Following that logic, Superadapter is my main proposal for a new name. > Maybe throw in OIDC (oidc-superadapter) if there's ever going to be a > saml-superadapter. > > Alternatively, we could focus on the lack of an adapter, with names > based on terms like Adapterless: > > - AKI: Adapterless Keycloak Integrator > - KOSA: Keycloak OIDC Sans-Adapter > - AKOS: Adapterless Keycloak OIDC Server > - KOAF: Keycloak OIDC Adapter-Free > - etc... > > Alex > > [1] > https://www.keycloak.org/docs/latest/securing_apps/index.html#what-are-client-adapters > > Quoting Bruno Oliveira (2018-08-20 09:54:42) > > Only to give a brief context for people not aware of it. Keycloak > > Generic Adapter was not well accepted, because the naming is too > > vague. So we have to reopen this discussion and think about a better > > naming. > > > > During our team call today I suggested just "keycloak-adapter", which > > would cover the apps which don't have its own specific adapter > > solution. > > > > That said, maybe we should open a new poll? I just created a new one > > where people can vote/suggest: > > > > https://poll.ly/#/Lbww4ebG > > _______________________________________________ > keycloak-user mailing list > keycloak-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-user > > _______________________________________________ > keycloak-user mailing list > keycloak-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-user > From gary.schulte at opengov.com Tue Aug 21 15:40:07 2018 From: gary.schulte at opengov.com (Gary Schulte) Date: Tue, 21 Aug 2018 12:40:07 -0700 Subject: [keycloak-dev] Keycloak 4.3 error column resourceen0_.uri does not exist Message-ID: We recently upgraded from 4.1.0.Final to 4.3.0.Final, and have been getting this exception in various operations: ... 102 more Caused by: org.postgresql.util.PSQLException: ERROR: column resourceen0_.uri does not exist Position: 336 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155) at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:118) at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:504) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:70) ... 120 more I first encountered this trying to export the configuration via `keycloak.migration.action=export` , but am now encountering it in a number of operations such as impersonating a user from the admin console. We aborted an upgrade to 4.2.1 due to a failed db migration, but 4.3.0 worked fine. Presumably there is a problem with the ddl migration. Any speculation on root cause and/or a fix? Is there a DDL published FYI we have keycloak deployed to k8s using an Amazon RDS postgres instance for the backing store. TIA Gary S From sthorger at redhat.com Tue Aug 21 15:52:22 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 21 Aug 2018 21:52:22 +0200 Subject: [keycloak-dev] Configurable signature algorithms Message-ID: Currently, Keycloak always use RS256 both for access tokens and id tokens. We're working on introducing support for more algorithms and the ability to change the default for a realm and also for a client. Now the question is should have we two options one for access token and another for ID token. Or just one for both? From slaskawi at redhat.com Tue Aug 21 21:52:26 2018 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 22 Aug 2018 03:52:26 +0200 Subject: [keycloak-dev] [keycloak-user] Keycloak Proxy Rename In-Reply-To: References: <20180625154229.GA17717@abstractj.org> <153479905785.6959.1361851594171681280@tyrfing> <1A3C52DFCD06494D8528644858247BF01C180537@EX10MBOX03.pnnl.gov> Message-ID: So if Keycloak Proxy is not an option (nor anything similar to that), I would also stay away from anything close to the "adapter". Therefore, +1 to Keycloak Gatekeeper (@Bruno, I changed my vote in the Polly). On Tue, Aug 21, 2018 at 11:58 PM Stian Thorgersen wrote: > Sure, proxy is the obvious name, but for reasons already mentioned by Bruno > it's not really an option for us. > > It comes from the Keycloak team, so it should have the Keycloak name in it. > I agree that doesn't automatically state it's a generic OIDC adapter, but > I'd like to keep our name in there. > > >From the suggestions so far there are two I like: > > * Keycloak Gatekeeper - suggested by Thomas on the poll. I really like this > and it fits nicely with Keycloak. It's also so much cooler than > proxy/standalone/etc. > * Keycloak Standalone Adapter > > On Tue, 21 Aug 2018 at 04:27, Fox, Kevin M wrote: > > > Coming from the outside world, I mostly searched for oidc and proxy as > > thats what I needed. I found oauth2_proxy after a little searching, but > > have been disappointed in how slow its releases are. Bugs aren't getting > > fixed quickly. When I looked at keycloak-proxy initially, I didn't look > > closer for a while as i thought is was keycloak specific. > > > > So, something like oidc-proxy might get you more successful hits. > > > > Thanks, > > Kevin > > ________________________________________ > > From: keycloak-user-bounces at lists.jboss.org [ > > keycloak-user-bounces at lists.jboss.org] on behalf of Alex Szczuczko [ > > aszczucz at redhat.com] > > Sent: Monday, August 20, 2018 2:04 PM > > To: Bruno Oliveira; Hynek Mlnarik > > Cc: keycloak-dev; keycloak-user > > Subject: Re: [keycloak-user] Keycloak Proxy Rename > > > > In thinking a new name, I tried to look hard at these things: > > > > 1. what this software actually does. > > > > 2. what makes this software desirable to a user. > > > > 3. what "adapter" has meant for keycloak in the past. > > > > I'm not the best person to answer these questions, but here's what I've > > dug up: > > > > 1. Accepts HTTP requests and talks with Keycloak via OIDC to see if > > the client it serves should treat the requests as authenticated > > and/or authorized. > > > > 2. It avoids the need to install a bit of Keycloak software into the > > users' applications. > > > > 3. According to the docs[1]: Keycloak client adapters are libraries > > that makes it very easy to secure applications and services with > > Keycloak ... our adapters easy to use and they require less > > boilerplate code than what is typically required by a library. > > > > #1 is what we've been focusing on with names like "proxy". The reasons > > such names are dissatisfying is there is nothing unique about sitting in > > between two endpoints and doing stuff. So, we need to look at what that > > "stuff" means for Keycloak. > > > > #3 in combination with #2 tells us what this "stuff" means for Keycloak. > > This new software is clearly not an adapter. Actually, this new software > > accomplishes the mission of an adapter better than adapters themselves! > > > > Following that logic, Superadapter is my main proposal for a new name. > > Maybe throw in OIDC (oidc-superadapter) if there's ever going to be a > > saml-superadapter. > > > > Alternatively, we could focus on the lack of an adapter, with names > > based on terms like Adapterless: > > > > - AKI: Adapterless Keycloak Integrator > > - KOSA: Keycloak OIDC Sans-Adapter > > - AKOS: Adapterless Keycloak OIDC Server > > - KOAF: Keycloak OIDC Adapter-Free > > - etc... > > > > Alex > > > > [1] > > > https://www.keycloak.org/docs/latest/securing_apps/index.html#what-are-client-adapters > > > > Quoting Bruno Oliveira (2018-08-20 09:54:42) > > > Only to give a brief context for people not aware of it. Keycloak > > > Generic Adapter was not well accepted, because the naming is too > > > vague. So we have to reopen this discussion and think about a better > > > naming. > > > > > > During our team call today I suggested just "keycloak-adapter", which > > > would cover the apps which don't have its own specific adapter > > > solution. > > > > > > That said, maybe we should open a new poll? I just created a new one > > > where people can vote/suggest: > > > > > > https://poll.ly/#/Lbww4ebG > > > > _______________________________________________ > > keycloak-user mailing list > > keycloak-user at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > > _______________________________________________ > > keycloak-user mailing list > > keycloak-user at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > _______________________________________________ > keycloak-user mailing list > keycloak-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-user > From slaskawi at redhat.com Tue Aug 21 21:55:23 2018 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 22 Aug 2018 03:55:23 +0200 Subject: [keycloak-dev] Configurable signature algorithms In-Reply-To: References: Message-ID: Setting them separately seems more flexible to me. On the other hand, it is hard for me to imagine a use case where a client would use two different signature algorithms... +1 for having two separate options. We can always set them equal in the Admin Console if we wish. On Wed, Aug 22, 2018 at 2:12 AM Stian Thorgersen wrote: > Currently, Keycloak always use RS256 both for access tokens and id tokens. > We're working on introducing support for more algorithms and the ability to > change the default for a realm and also for a client. > > Now the question is should have we two options one for access token and > another for ID token. Or just one for both? > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Wed Aug 22 02:45:48 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 22 Aug 2018 08:45:48 +0200 Subject: [keycloak-dev] Configurable signature algorithms In-Reply-To: References: Message-ID: The use-case for separate is if the front-end app that is using id token uses one algorithm, while back-ends who are using the access token expects a different algorithm. Now, the question is how likely is that. On Wed, 22 Aug 2018, 03:55 Sebastian Laskawiec, wrote: > Setting them separately seems more flexible to me. On the other hand, it > is hard for me to imagine a use case where a client would use two different > signature algorithms... > > +1 for having two separate options. We can always set them equal in the > Admin Console if we wish. > > On Wed, Aug 22, 2018 at 2:12 AM Stian Thorgersen > wrote: > >> Currently, Keycloak always use RS256 both for access tokens and id tokens. >> We're working on introducing support for more algorithms and the ability >> to >> change the default for a realm and also for a client. >> >> Now the question is should have we two options one for access token and >> another for ID token. Or just one for both? >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > From mposolda at redhat.com Wed Aug 22 03:17:32 2018 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 22 Aug 2018 09:17:32 +0200 Subject: [keycloak-dev] Configurable signature algorithms In-Reply-To: References: Message-ID: <9f345d89-2567-a973-13fb-ab3d3d127091@redhat.com> Not sure how likely is that. Maybe we can have just one option and introduce the other once there is use-case for that? The OIDC specification has a way to use separate signature/encryption algorithms for ID tokens, UserInfo response and "request" object - http://openid.net/specs/openid-connect-registration-1_0.html#ClientMetadata . So it's possible that at least for those we may need an ability to add separate algorithms to conform the specs? Marek On 22/08/18 08:45, Stian Thorgersen wrote: > The use-case for separate is if the front-end app that is using id token > uses one algorithm, while back-ends who are using the access token expects > a different algorithm. Now, the question is how likely is that. > > On Wed, 22 Aug 2018, 03:55 Sebastian Laskawiec, wrote: > >> Setting them separately seems more flexible to me. On the other hand, it >> is hard for me to imagine a use case where a client would use two different >> signature algorithms... >> >> +1 for having two separate options. We can always set them equal in the >> Admin Console if we wish. >> >> On Wed, Aug 22, 2018 at 2:12 AM Stian Thorgersen >> wrote: >> >>> Currently, Keycloak always use RS256 both for access tokens and id tokens. >>> We're working on introducing support for more algorithms and the ability >>> to >>> change the default for a realm and also for a client. >>> >>> Now the question is should have we two options one for access token and >>> another for ID token. Or just one for both? >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From jlieskov at redhat.com Wed Aug 22 03:23:20 2018 From: jlieskov at redhat.com (Jan Lieskovsky) Date: Wed, 22 Aug 2018 09:23:20 +0200 Subject: [keycloak-dev] Configurable signature algorithms In-Reply-To: References: Message-ID: Per this explanation the id_token is meant for the specific client only (is signed with a secret that is known to that client). On the other hand (from that article), the API / access_token can be of any type (not necessarily a JWT based one), and it is expected to have audience set to API's unique identifier. To me this sounds, like certain clients might want to choose just some specific algorithm to share the secret known to them with the frontend. Something like choosing specific protocol version (TLS v1.0, TLS v1.1, TLS v1.2 etc.) in the case of SSL/TLS connections. It might be possible some clients wouldn't support all signature algorithms supported / implemented by Keycloak server. So having id_token / access_token signature algorithms configurable, would allow them to choose the one, they are able to communicate under. I agree that in majority of the cases, the algorithms in both cases would be the same. But having the option to specify different algorithms for id and access tokens, looks to me more clients might be able to connect / use the services, compared with the case when just one identical algorithm would be enforced / used for both type of tokens. On Wed, Aug 22, 2018 at 8:45 AM, Stian Thorgersen wrote: > The use-case for separate is if the front-end app that is using id token > uses one algorithm, while back-ends who are using the access token expects > a different algorithm. Now, the question is how likely is that. > > On Wed, 22 Aug 2018, 03:55 Sebastian Laskawiec, > wrote: > > > Setting them separately seems more flexible to me. On the other hand, it > > is hard for me to imagine a use case where a client would use two > different > > signature algorithms... > > > > +1 for having two separate options. We can always set them equal in the > > Admin Console if we wish. > > > > On Wed, Aug 22, 2018 at 2:12 AM Stian Thorgersen > > wrote: > > > >> Currently, Keycloak always use RS256 both for access tokens and id > tokens. > >> We're working on introducing support for more algorithms and the ability > >> to > >> change the default for a realm and also for a client. > >> > >> Now the question is should have we two options one for access token and > >> another for ID token. Or just one for both? > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From mposolda at redhat.com Wed Aug 22 03:47:34 2018 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 22 Aug 2018 09:47:34 +0200 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: +1 for the LocaleSelectorSPI. Now I recall that on the previous project I was working few years ago (GateIn Portal) we had such flexibility due the people having different requirements on how the locale should be chosen :) Regarding your proposal, It looks like a good start. But maybe I would rather remove the class LocaleHelper entirely and extract the stuff directly to the DefaultLocaleSelectorProvider? This will have an advantage that DefaultLocaleSelectorProvider may have various protected methods (EG. like getLocaleFromCookie, getLocaleFromUiLocalesParameter, getLocaleFromUserProfile etc), so that if someone wants to override default behaviour, he doesn't need to extract whole code, but just extend from DefaultLocaleSelectorProvider and call protected methods directly etc? Also I wonder if DefaultLocaleSelectorProvider should have some configuration option, which will allow to easily specify default order (EG. "cookie kclocale_param profile ui_locale_param accept_header realm" ) so that 90% of people can just configure the value in standalone.xml without a need to implement their own LocaleSelectorProvider? Marek On 15/08/18 18:13, Johannes Knutsen wrote: > I did some quick work on extracting an LocaleSelectorSPI. Please have > a look at https://github.com/keycloak/keycloak/compare/master...knutz3n:feature/locale-spi. > Thoughts? > > - Johannes > > On Tue, Aug 14, 2018 at 8:32 PM, Stian Thorgersen wrote: >> The most common case is that the same user uses the browser. So if the user >> has actively changed the locale in Keycloak it should stay like that. I >> would actually suggest that if the user has changed the locale in Keycloak, >> you should have the locale added to the token so the application can update >> it's locale. That would result in all applications in the realm using the >> same locale for the user. >> >> Now if we change it to what you are suggesting you can end up with >> situations where the first login page is displayed in one language, then >> once the user has entered username/email it would end up switching to the >> locale associated with the user. >> >> If you really don't like that behaviour I would suggest we either introduce >> a LocaleSelectorSPI where you can change the logic with a custom provider. >> Alternatively, we could potentially add an option on the realm to be able to >> disable the locale selector on Keycloak, which means that the user wouldn't >> be able to change the locale and ui_locales would always take precedence. In >> either case I think it's best to keep the current behaviour as the default, >> with some way of being able to change it. >> >> On Tue, 14 Aug 2018 at 12:56, Johannes Knutsen wrote: >>> It currently works the way you describe it. The ui_locales parameter >>> is taking effect as long as the locale cookie isn't set. However, the >>> logic implies that you always expect the same user to authenticate and >>> our preference is that the user should get a login page matching the >>> locale of the client which sends the user to the login page, if the >>> client specified preferred locales as an ui_locales parameter. >>> Using the locale cookie to override the ui_locales parameter makes the >>> user experience inconsistent when moving from a client with a locale >>> different from the last logged in user's preferred locale. >>> >>> A proposed change is to change locale priority as follows: >>> 1. kc_locale query parameter. This updates the user's locale and locale >>> cookie. >>> 2. user profile locale if a user is authenticated. This updates the >>> locale cookie. >>> 3. ui_locales query parameter. If ui_locales contains the locale >>> cookie value, return it. Otherwise, return the first locale available >>> from ui_locales >>> 4. locale cookie >>> 5. accept language http header >>> >>> Is this change something you would consider merging if I create a pull >>> request? Any other adjustments or thoughts regarding such a change? >>> >>> - Johannes >>> >>> On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen >>> wrote: >>>> The way at least it was supposed to work was that the Locale cookie >>>> would >>>> only be set if a user has explicitly changed the locale. That way we can >>>> remember a users preference even when the user is logged out. >>>> >>>> Are you seeing ui_locales not taking affect without a user having used >>>> the >>>> locale drop-down? >>>> >>>> On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen >>>> wrote: >>>>> Currently the LocaleHelper#getUserLocale chooses locale priority based >>>>> on the following order: >>>>> >>>>> 1. kc_locale query parameter >>>>> 2. Locale cookie >>>>> 3. User profile >>>>> 4. ui_locales query parameter >>>>> 5. Accept-Language http header >>>>> >>>>> We are building clients which uses the ui_locales parameter to select >>>>> localization of the login to match the locale of the client. >>>>> Since the locale cookie has a higher priority than the ui_locales >>>>> query parameter, the ui_locales query parameter is ignored for any >>>>> auth requests with a kc_locale cookie present. >>>>> >>>>> 1. Do you see any implications of prioritizing the ui_locales >>>>> parameter above the locale cookie when resolving locale? >>>>> 2. Are there reasons why the ui_locales does not update the locale >>>>> cookie value the same way kc_locale parameter does it? >>>>> >>>>> Best regards, >>>>> Johannes Knutsen >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From jlieskov at redhat.com Wed Aug 22 04:51:44 2018 From: jlieskov at redhat.com (Jan Lieskovsky) Date: Wed, 22 Aug 2018 10:51:44 +0200 Subject: [keycloak-dev] [keycloak-user] Keycloak Proxy Rename In-Reply-To: References: <20180625154229.GA17717@abstractj.org> <153479905785.6959.1361851594171681280@tyrfing> <1A3C52DFCD06494D8528644858247BF01C180537@EX10MBOX03.pnnl.gov> Message-ID: Added keycloak-envoy, keycloak-intermediary, and keycloak-mediator for the considerations there yet (unsure though having keycloak-envoy wouldn't induce same feature expectations [thus load balancing, fault injection etc.] with istio-envoy for those familiar with istio). On Wed, Aug 22, 2018 at 3:52 AM, Sebastian Laskawiec wrote: > So if Keycloak Proxy is not an option (nor anything similar to that), I > would also stay away from anything close to the "adapter". > > Therefore, +1 to Keycloak Gatekeeper (@Bruno, I changed my vote in the > Polly). > > On Tue, Aug 21, 2018 at 11:58 PM Stian Thorgersen > wrote: > > > Sure, proxy is the obvious name, but for reasons already mentioned by > Bruno > > it's not really an option for us. > > > > It comes from the Keycloak team, so it should have the Keycloak name in > it. > > I agree that doesn't automatically state it's a generic OIDC adapter, but > > I'd like to keep our name in there. > > > > >From the suggestions so far there are two I like: > > > > * Keycloak Gatekeeper - suggested by Thomas on the poll. I really like > this > > and it fits nicely with Keycloak. It's also so much cooler than > > proxy/standalone/etc. > > * Keycloak Standalone Adapter > > > > On Tue, 21 Aug 2018 at 04:27, Fox, Kevin M wrote: > > > > > Coming from the outside world, I mostly searched for oidc and proxy as > > > thats what I needed. I found oauth2_proxy after a little searching, but > > > have been disappointed in how slow its releases are. Bugs aren't > getting > > > fixed quickly. When I looked at keycloak-proxy initially, I didn't look > > > closer for a while as i thought is was keycloak specific. > > > > > > So, something like oidc-proxy might get you more successful hits. > > > > > > Thanks, > > > Kevin > > > ________________________________________ > > > From: keycloak-user-bounces at lists.jboss.org [ > > > keycloak-user-bounces at lists.jboss.org] on behalf of Alex Szczuczko [ > > > aszczucz at redhat.com] > > > Sent: Monday, August 20, 2018 2:04 PM > > > To: Bruno Oliveira; Hynek Mlnarik > > > Cc: keycloak-dev; keycloak-user > > > Subject: Re: [keycloak-user] Keycloak Proxy Rename > > > > > > In thinking a new name, I tried to look hard at these things: > > > > > > 1. what this software actually does. > > > > > > 2. what makes this software desirable to a user. > > > > > > 3. what "adapter" has meant for keycloak in the past. > > > > > > I'm not the best person to answer these questions, but here's what I've > > > dug up: > > > > > > 1. Accepts HTTP requests and talks with Keycloak via OIDC to see if > > > the client it serves should treat the requests as authenticated > > > and/or authorized. > > > > > > 2. It avoids the need to install a bit of Keycloak software into > the > > > users' applications. > > > > > > 3. According to the docs[1]: Keycloak client adapters are libraries > > > that makes it very easy to secure applications and services with > > > Keycloak ... our adapters easy to use and they require less > > > boilerplate code than what is typically required by a library. > > > > > > #1 is what we've been focusing on with names like "proxy". The reasons > > > such names are dissatisfying is there is nothing unique about sitting > in > > > between two endpoints and doing stuff. So, we need to look at what that > > > "stuff" means for Keycloak. > > > > > > #3 in combination with #2 tells us what this "stuff" means for > Keycloak. > > > This new software is clearly not an adapter. Actually, this new > software > > > accomplishes the mission of an adapter better than adapters themselves! > > > > > > Following that logic, Superadapter is my main proposal for a new name. > > > Maybe throw in OIDC (oidc-superadapter) if there's ever going to be a > > > saml-superadapter. > > > > > > Alternatively, we could focus on the lack of an adapter, with names > > > based on terms like Adapterless: > > > > > > - AKI: Adapterless Keycloak Integrator > > > - KOSA: Keycloak OIDC Sans-Adapter > > > - AKOS: Adapterless Keycloak OIDC Server > > > - KOAF: Keycloak OIDC Adapter-Free > > > - etc... > > > > > > Alex > > > > > > [1] > > > > > https://www.keycloak.org/docs/latest/securing_apps/index. > html#what-are-client-adapters > > > > > > Quoting Bruno Oliveira (2018-08-20 09:54:42) > > > > Only to give a brief context for people not aware of it. Keycloak > > > > Generic Adapter was not well accepted, because the naming is too > > > > vague. So we have to reopen this discussion and think about a better > > > > naming. > > > > > > > > During our team call today I suggested just "keycloak-adapter", which > > > > would cover the apps which don't have its own specific adapter > > > > solution. > > > > > > > > That said, maybe we should open a new poll? I just created a new one > > > > where people can vote/suggest: > > > > > > > > https://poll.ly/#/Lbww4ebG > > > > > > _______________________________________________ > > > keycloak-user mailing list > > > keycloak-user at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > > > > _______________________________________________ > > > keycloak-user mailing list > > > keycloak-user at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > > > _______________________________________________ > > keycloak-user mailing list > > keycloak-user at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From pskopek at redhat.com Wed Aug 22 04:52:26 2018 From: pskopek at redhat.com (Peter Skopek) Date: Wed, 22 Aug 2018 10:52:26 +0200 Subject: [keycloak-dev] [keycloak-user] Keycloak Proxy Rename In-Reply-To: References: <20180625154229.GA17717@abstractj.org> <153479905785.6959.1361851594171681280@tyrfing> <1A3C52DFCD06494D8528644858247BF01C180537@EX10MBOX03.pnnl.gov> Message-ID: +1 I like Keycloak Gatekeeper too. Peter On Wed, Aug 22, 2018 at 6:15 AM Sebastian Laskawiec wrote: > > So if Keycloak Proxy is not an option (nor anything similar to that), I > would also stay away from anything close to the "adapter". > > Therefore, +1 to Keycloak Gatekeeper (@Bruno, I changed my vote in the > Polly). > > On Tue, Aug 21, 2018 at 11:58 PM Stian Thorgersen > wrote: > > > Sure, proxy is the obvious name, but for reasons already mentioned by Bruno > > it's not really an option for us. > > > > It comes from the Keycloak team, so it should have the Keycloak name in it. > > I agree that doesn't automatically state it's a generic OIDC adapter, but > > I'd like to keep our name in there. > > > > >From the suggestions so far there are two I like: > > > > * Keycloak Gatekeeper - suggested by Thomas on the poll. I really like this > > and it fits nicely with Keycloak. It's also so much cooler than > > proxy/standalone/etc. > > * Keycloak Standalone Adapter > > > > On Tue, 21 Aug 2018 at 04:27, Fox, Kevin M wrote: > > > > > Coming from the outside world, I mostly searched for oidc and proxy as > > > thats what I needed. I found oauth2_proxy after a little searching, but > > > have been disappointed in how slow its releases are. Bugs aren't getting > > > fixed quickly. When I looked at keycloak-proxy initially, I didn't look > > > closer for a while as i thought is was keycloak specific. > > > > > > So, something like oidc-proxy might get you more successful hits. > > > > > > Thanks, > > > Kevin > > > ________________________________________ > > > From: keycloak-user-bounces at lists.jboss.org [ > > > keycloak-user-bounces at lists.jboss.org] on behalf of Alex Szczuczko [ > > > aszczucz at redhat.com] > > > Sent: Monday, August 20, 2018 2:04 PM > > > To: Bruno Oliveira; Hynek Mlnarik > > > Cc: keycloak-dev; keycloak-user > > > Subject: Re: [keycloak-user] Keycloak Proxy Rename > > > > > > In thinking a new name, I tried to look hard at these things: > > > > > > 1. what this software actually does. > > > > > > 2. what makes this software desirable to a user. > > > > > > 3. what "adapter" has meant for keycloak in the past. > > > > > > I'm not the best person to answer these questions, but here's what I've > > > dug up: > > > > > > 1. Accepts HTTP requests and talks with Keycloak via OIDC to see if > > > the client it serves should treat the requests as authenticated > > > and/or authorized. > > > > > > 2. It avoids the need to install a bit of Keycloak software into the > > > users' applications. > > > > > > 3. According to the docs[1]: Keycloak client adapters are libraries > > > that makes it very easy to secure applications and services with > > > Keycloak ... our adapters easy to use and they require less > > > boilerplate code than what is typically required by a library. > > > > > > #1 is what we've been focusing on with names like "proxy". The reasons > > > such names are dissatisfying is there is nothing unique about sitting in > > > between two endpoints and doing stuff. So, we need to look at what that > > > "stuff" means for Keycloak. > > > > > > #3 in combination with #2 tells us what this "stuff" means for Keycloak. > > > This new software is clearly not an adapter. Actually, this new software > > > accomplishes the mission of an adapter better than adapters themselves! > > > > > > Following that logic, Superadapter is my main proposal for a new name. > > > Maybe throw in OIDC (oidc-superadapter) if there's ever going to be a > > > saml-superadapter. > > > > > > Alternatively, we could focus on the lack of an adapter, with names > > > based on terms like Adapterless: > > > > > > - AKI: Adapterless Keycloak Integrator > > > - KOSA: Keycloak OIDC Sans-Adapter > > > - AKOS: Adapterless Keycloak OIDC Server > > > - KOAF: Keycloak OIDC Adapter-Free > > > - etc... > > > > > > Alex > > > > > > [1] > > > > > https://www.keycloak.org/docs/latest/securing_apps/index.html#what-are-client-adapters > > > > > > Quoting Bruno Oliveira (2018-08-20 09:54:42) > > > > Only to give a brief context for people not aware of it. Keycloak > > > > Generic Adapter was not well accepted, because the naming is too > > > > vague. So we have to reopen this discussion and think about a better > > > > naming. > > > > > > > > During our team call today I suggested just "keycloak-adapter", which > > > > would cover the apps which don't have its own specific adapter > > > > solution. > > > > > > > > That said, maybe we should open a new poll? I just created a new one > > > > where people can vote/suggest: > > > > > > > > https://poll.ly/#/Lbww4ebG > > > > > > _______________________________________________ > > > keycloak-user mailing list > > > keycloak-user at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > > > > _______________________________________________ > > > keycloak-user mailing list > > > keycloak-user at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > > > _______________________________________________ > > keycloak-user mailing list > > keycloak-user at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-user > > > _______________________________________________ > keycloak-user mailing list > keycloak-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-user From alistair.doswald at elca.ch Wed Aug 22 05:12:38 2018 From: alistair.doswald at elca.ch (Doswald Alistair) Date: Wed, 22 Aug 2018 09:12:38 +0000 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Message-ID: Hello Thomas, As you (and a couple of others) have suggested, I?ve done a PR of the mapper code along with a few tests. Concerning the GO client, the name is actually misleading. It?s in fact more of an administration bridge: part of an interface that we set in front of keycloak. Cheers, Alistair From: Thomas Darimont Sent: mardi 14 ao?t 2018 21:26 To: Doswald Alistair Cc: keycloak-dev Subject: Re: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project Hello Alistair, those are IMHO awesome modules thanks for sharing :) btw. you also have a handy go Keycloak client :) https://github.com/cloudtrust/keycloak-client Regarding SAML ScriptMapper (KEYCLOAK-5520) I think it totally makes sense to integrate that into Keycloak directly. I was onto writing that myself but then priorities changed..., but your implementation looks quite good already :) I'm pretty sure that if you get the tests running inside the Keycloak test-suite the Keycloak team would be happy to discuss/merge your PR. Cheers, Thomas Am Di., 14. Aug. 2018 um 12:04 Uhr schrieb Doswald Alistair >: Hello, I just wanted to let this mailing list know that for the Cloudtrust project (https://github.com/cloudtrust), we have developed a certain number modules for Keycloak. These are currently compatible with the version 3.4.3.Final of Keycloak, but we will make them compatible with Keycloak 4.X (where X will be the latest sub-version of Keycloak when we start working on this) as soon as we can. These modules are: * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an implementation of the WS-Federation protocol for keycloak. This allows to select the WS-Federation protocol for Keycloak clients and for identity brokers. * keycloak-authorization (https://github.com/cloudtrust/keycloak-authorization): this module allows the use of the client authorization system to prevent a user which is authenticated in a Keycloak realm to access a given client. It works no matter which protocol is used, and without the client having to support any extra protocol. Note: this solution is a bit hacky, but necessary for one of our use-cases. * keycloak-client-mappers (https://github.com/cloudtrust/keycloak-client-mappers): a module for adding any mappers that we might need that are not yet part of Keycloak. Currently only contains a JavaScript mapper for SAML, analogous to the OIDC script mapper. I've noticed that there's an open issue for this feature (https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could submit this code not as a module but a solution to the issue. * keycloak-export (https://github.com/cloudtrust/keycloak-export): a module adding an endpoint to fully export a realm while Keycloak is still running (no need for restarts!). Cheers, Alistair PS: I mailed this to both dev and user mailing lists as I believe it may interest members of both mailing lists. However, upon sending to the dev mailing list the first time it bounced. This is the second attempt. _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From sthorger at redhat.com Wed Aug 22 05:54:04 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 22 Aug 2018 11:54:04 +0200 Subject: [keycloak-dev] Configurable signature algorithms In-Reply-To: References: Message-ID: What I'm thinking now is to have a default singing algorithm for the realm, but for a client have the option to specify ID token, user info and access token algorithms separately. Most clients won't have to touch those settings at all and can just rely on the realm default. I don't think there's a need to have separate for a realm. On Wed, 22 Aug 2018 at 09:23, Jan Lieskovsky wrote: > Per this explanation > > the id_token is meant for the specific client only (is signed with a > secret that is > known to that client). On the other hand (from that article), the API / > access_token can be of any type > (not necessarily a JWT based one), and it is expected to have audience set > to API's unique identifier. > > To me this sounds, like certain clients might want to choose just some > specific algorithm to share > the secret known to them with the frontend. Something like choosing > specific protocol version > (TLS v1.0, TLS v1.1, TLS v1.2 etc.) in the case of SSL/TLS connections. It > might be possible some > clients wouldn't support all signature algorithms supported / implemented > by Keycloak server. So having > id_token / access_token signature algorithms configurable, would allow > them to choose the one, they > are able to communicate under. I agree that in majority of the cases, the > algorithms in both cases would > be the same. But having the option to specify different algorithms for id > and access tokens, looks to me > more clients might be able to connect / use the services, compared with > the case when just one identical > algorithm would be enforced / used for both type of tokens. > > > On Wed, Aug 22, 2018 at 8:45 AM, Stian Thorgersen > wrote: > >> The use-case for separate is if the front-end app that is using id token >> uses one algorithm, while back-ends who are using the access token expects >> a different algorithm. Now, the question is how likely is that. >> >> On Wed, 22 Aug 2018, 03:55 Sebastian Laskawiec, >> wrote: >> >> > Setting them separately seems more flexible to me. On the other hand, it >> > is hard for me to imagine a use case where a client would use two >> different >> > signature algorithms... >> > >> > +1 for having two separate options. We can always set them equal in the >> > Admin Console if we wish. >> > >> > On Wed, Aug 22, 2018 at 2:12 AM Stian Thorgersen >> > wrote: >> > >> >> Currently, Keycloak always use RS256 both for access tokens and id >> tokens. >> >> We're working on introducing support for more algorithms and the >> ability >> >> to >> >> change the default for a realm and also for a client. >> >> >> >> Now the question is should have we two options one for access token and >> >> another for ID token. Or just one for both? >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > From ssilvert at redhat.com Wed Aug 22 07:59:02 2018 From: ssilvert at redhat.com (Stan Silvert) Date: Wed, 22 Aug 2018 07:59:02 -0400 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: References: Message-ID: <70b769f9-af6d-ec50-bac7-413d3bfe4994@redhat.com> On 8/22/2018 3:47 AM, Marek Posolda wrote: > +1 for the LocaleSelectorSPI. Now I recall that on the previous project > I was working few years ago (GateIn Portal) we had such flexibility due > the people having different requirements on how the locale should be > chosen :) > > Regarding your proposal, It looks like a good start. But maybe I would > rather remove the class LocaleHelper entirely and extract the stuff > directly to the DefaultLocaleSelectorProvider? This will have an > advantage that DefaultLocaleSelectorProvider may have various protected > methods (EG. like getLocaleFromCookie, getLocaleFromUiLocalesParameter, > getLocaleFromUserProfile etc), so that if someone wants to override > default behaviour, he doesn't need to extract whole code, but just > extend from DefaultLocaleSelectorProvider and call protected methods > directly etc? > > Also I wonder if DefaultLocaleSelectorProvider should have some > configuration option, which will allow to easily specify default order > (EG. "cookie kclocale_param profile ui_locale_param accept_header realm" > ) so that 90% of people can just configure the value in standalone.xml > without a need to implement their own LocaleSelectorProvider? -1 to configuration via standalone.xml.? In general, I don't think we should create a simplified interface to address an edge case. That just introduces ongoing maintenance.? What we have now gets it right 99.9% of the time. +1 to LocaleSelectorSPI.? Even if we did the standalone.xml thing, somebody would still need LocaleSelectorSPI for complex cases. > > Marek > > On 15/08/18 18:13, Johannes Knutsen wrote: >> I did some quick work on extracting an LocaleSelectorSPI. Please have >> a look at https://github.com/keycloak/keycloak/compare/master...knutz3n:feature/locale-spi. >> Thoughts? >> >> - Johannes >> >> On Tue, Aug 14, 2018 at 8:32 PM, Stian Thorgersen wrote: >>> The most common case is that the same user uses the browser. So if the user >>> has actively changed the locale in Keycloak it should stay like that. I >>> would actually suggest that if the user has changed the locale in Keycloak, >>> you should have the locale added to the token so the application can update >>> it's locale. That would result in all applications in the realm using the >>> same locale for the user. >>> >>> Now if we change it to what you are suggesting you can end up with >>> situations where the first login page is displayed in one language, then >>> once the user has entered username/email it would end up switching to the >>> locale associated with the user. >>> >>> If you really don't like that behaviour I would suggest we either introduce >>> a LocaleSelectorSPI where you can change the logic with a custom provider. >>> Alternatively, we could potentially add an option on the realm to be able to >>> disable the locale selector on Keycloak, which means that the user wouldn't >>> be able to change the locale and ui_locales would always take precedence. In >>> either case I think it's best to keep the current behaviour as the default, >>> with some way of being able to change it. >>> >>> On Tue, 14 Aug 2018 at 12:56, Johannes Knutsen wrote: >>>> It currently works the way you describe it. The ui_locales parameter >>>> is taking effect as long as the locale cookie isn't set. However, the >>>> logic implies that you always expect the same user to authenticate and >>>> our preference is that the user should get a login page matching the >>>> locale of the client which sends the user to the login page, if the >>>> client specified preferred locales as an ui_locales parameter. >>>> Using the locale cookie to override the ui_locales parameter makes the >>>> user experience inconsistent when moving from a client with a locale >>>> different from the last logged in user's preferred locale. >>>> >>>> A proposed change is to change locale priority as follows: >>>> 1. kc_locale query parameter. This updates the user's locale and locale >>>> cookie. >>>> 2. user profile locale if a user is authenticated. This updates the >>>> locale cookie. >>>> 3. ui_locales query parameter. If ui_locales contains the locale >>>> cookie value, return it. Otherwise, return the first locale available >>>> from ui_locales >>>> 4. locale cookie >>>> 5. accept language http header >>>> >>>> Is this change something you would consider merging if I create a pull >>>> request? Any other adjustments or thoughts regarding such a change? >>>> >>>> - Johannes >>>> >>>> On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen >>>> wrote: >>>>> The way at least it was supposed to work was that the Locale cookie >>>>> would >>>>> only be set if a user has explicitly changed the locale. That way we can >>>>> remember a users preference even when the user is logged out. >>>>> >>>>> Are you seeing ui_locales not taking affect without a user having used >>>>> the >>>>> locale drop-down? >>>>> >>>>> On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen >>>>> wrote: >>>>>> Currently the LocaleHelper#getUserLocale chooses locale priority based >>>>>> on the following order: >>>>>> >>>>>> 1. kc_locale query parameter >>>>>> 2. Locale cookie >>>>>> 3. User profile >>>>>> 4. ui_locales query parameter >>>>>> 5. Accept-Language http header >>>>>> >>>>>> We are building clients which uses the ui_locales parameter to select >>>>>> localization of the login to match the locale of the client. >>>>>> Since the locale cookie has a higher priority than the ui_locales >>>>>> query parameter, the ui_locales query parameter is ignored for any >>>>>> auth requests with a kc_locale cookie present. >>>>>> >>>>>> 1. Do you see any implications of prioritizing the ui_locales >>>>>> parameter above the locale cookie when resolving locale? >>>>>> 2. Are there reasons why the ui_locales does not update the locale >>>>>> cookie value the same way kc_locale parameter does it? >>>>>> >>>>>> Best regards, >>>>>> Johannes Knutsen >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From psilva at redhat.com Wed Aug 22 08:17:47 2018 From: psilva at redhat.com (Pedro Igor Silva) Date: Wed, 22 Aug 2018 09:17:47 -0300 Subject: [keycloak-dev] Keycloak 4.3 error column resourceen0_.uri does not exist In-Reply-To: References: Message-ID: Hi, This column was dropped in favor of a new table holding multiple uris for a resource. It seems you are still using old entity format (see ResourceEntity) which does not have a "uri" field anymore. Although your database looks correct. Regards. Pedro Igor On Tue, Aug 21, 2018 at 4:40 PM, Gary Schulte wrote: > We recently upgraded from 4.1.0.Final to 4.3.0.Final, and have been getting > this exception in various operations: > > ... 102 more > Caused by: org.postgresql.util.PSQLException: ERROR: column > resourceen0_.uri does not exist > Position: 336 > at > org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse( > QueryExecutorImpl.java:2433) > at > org.postgresql.core.v3.QueryExecutorImpl.processResults( > QueryExecutorImpl.java:2178) > at > org.postgresql.core.v3.QueryExecutorImpl.execute( > QueryExecutorImpl.java:306) > at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) > at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) > at > org.postgresql.jdbc.PgPreparedStatement.executeWithFlags( > PgPreparedStatement.java:155) > at > org.postgresql.jdbc.PgPreparedStatement.executeQuery( > PgPreparedStatement.java:118) > at > org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery( > WrappedPreparedStatement.java:504) > at > org.hibernate.engine.jdbc.internal.ResultSetReturnImpl. > extract(ResultSetReturnImpl.java:70) > ... 120 more > > > I first encountered this trying to export the configuration > via `keycloak.migration.action=export` , but am now encountering it in a > number of operations such as impersonating a user from the admin console. > > We aborted an upgrade to 4.2.1 due to a failed db migration, but 4.3.0 > worked fine. Presumably there is a problem with the ddl migration. > > Any speculation on root cause and/or a fix? Is there a DDL published > > FYI we have keycloak deployed to k8s using an Amazon RDS postgres instance > for the backing store. > > TIA > > Gary S > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Wed Aug 22 09:35:35 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 22 Aug 2018 15:35:35 +0200 Subject: [keycloak-dev] Keycloak master Docker image Message-ID: If you ever wanted to try something in Keycloak master without building it yourself you can now use the jboss/keycloak:master tag. It is automatically rebuilt every time there is a merge to master. From hmlnarik at redhat.com Wed Aug 22 10:48:50 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Wed, 22 Aug 2018 16:48:50 +0200 Subject: [keycloak-dev] Configurable signature algorithms In-Reply-To: References: Message-ID: +1 We could do it similarly for SAML clients (default set in realm, client can override if required) to have consistent experience. Indeed the set of algorithms would be different. On Wed, Aug 22, 2018 at 11:55 AM Stian Thorgersen wrote: > What I'm thinking now is to have a default singing algorithm for the realm, > but for a client have the option to specify ID token, user info and access > token algorithms separately. Most clients won't have to touch those > settings at all and can just rely on the realm default. I don't think > there's a need to have separate for a realm. > > On Wed, 22 Aug 2018 at 09:23, Jan Lieskovsky wrote: > > > Per this explanation > > > > the id_token is meant for the specific client only (is signed with a > > secret that is > > known to that client). On the other hand (from that article), the API / > > access_token can be of any type > > (not necessarily a JWT based one), and it is expected to have audience > set > > to API's unique identifier. > > > > To me this sounds, like certain clients might want to choose just some > > specific algorithm to share > > the secret known to them with the frontend. Something like choosing > > specific protocol version > > (TLS v1.0, TLS v1.1, TLS v1.2 etc.) in the case of SSL/TLS connections. > It > > might be possible some > > clients wouldn't support all signature algorithms supported / implemented > > by Keycloak server. So having > > id_token / access_token signature algorithms configurable, would allow > > them to choose the one, they > > are able to communicate under. I agree that in majority of the cases, the > > algorithms in both cases would > > be the same. But having the option to specify different algorithms for id > > and access tokens, looks to me > > more clients might be able to connect / use the services, compared with > > the case when just one identical > > algorithm would be enforced / used for both type of tokens. > > > > > > On Wed, Aug 22, 2018 at 8:45 AM, Stian Thorgersen > > wrote: > > > >> The use-case for separate is if the front-end app that is using id token > >> uses one algorithm, while back-ends who are using the access token > expects > >> a different algorithm. Now, the question is how likely is that. > >> > >> On Wed, 22 Aug 2018, 03:55 Sebastian Laskawiec, > >> wrote: > >> > >> > Setting them separately seems more flexible to me. On the other hand, > it > >> > is hard for me to imagine a use case where a client would use two > >> different > >> > signature algorithms... > >> > > >> > +1 for having two separate options. We can always set them equal in > the > >> > Admin Console if we wish. > >> > > >> > On Wed, Aug 22, 2018 at 2:12 AM Stian Thorgersen > > >> > wrote: > >> > > >> >> Currently, Keycloak always use RS256 both for access tokens and id > >> tokens. > >> >> We're working on introducing support for more algorithms and the > >> ability > >> >> to > >> >> change the default for a realm and also for a client. > >> >> > >> >> Now the question is should have we two options one for access token > and > >> >> another for ID token. Or just one for both? > >> >> _______________________________________________ > >> >> keycloak-dev mailing list > >> >> keycloak-dev at lists.jboss.org > >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> >> > >> > > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Wed Aug 22 13:29:25 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 22 Aug 2018 17:29:25 +0000 Subject: [keycloak-dev] ./target/dependency/log4j.properties Message-ID: Anyone know what this file is doing there? It seems to be a copy of ./src/test/resources/log4j.properties It seems this is copied when you run mvn build, but that results in changes to ./src/test/resources/log4j.properties not taking affect when you're working from the IDE. From sthorger at redhat.com Wed Aug 22 15:25:17 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 22 Aug 2018 19:25:17 +0000 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: That's quite worrying. Is it limited to browsing users through the admin console or are you seeing bad behaviour elsewhere? Same question applies to the heap. My best bet here is that it has that something has changed around user querying/caching. On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hello Keycloak team, > > has anyone encountered some performance issues after upgrading 3.4.3 to 4.x > (4.3.0)? > > Today I noticed a performance regression while preparing an upgrade > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. > > In our test environment, we have around ~100k test users stored in a > postgres-backed database. When we started the server with the new Keycloak > version, the migration went through, and everything looked fine at first, > but when we tried to browse the list of users via the admin-console, we > noticed that the CPU and memory consumption of the server increased > significantly, up to a point where Keycloak crashed with an OOME. > > All previous Keycloak versions including 3.4.3 were very modest with their > memory requirements and quite happy with ~1g heap. > However, that seems to have changed in Keycloak 4.3.0 - there we needed at > least 4g to prevent Keycloak from crashing with an OOME. > Furthermore, we noticed that the response times for browsing the paginated > user view increased significantly as well: > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation takes > ~7 seconds for a test realm with just 10k users. > > In the test realm with 100k users, the time to load a single page in the > users listing was 66 seconds for version 4.3.0, on average - compared to > quite stable 80ms in 3.4.3. > > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 seconds > for 100k users, so I assume the processing logic in Keycloak is the > culprit. > > The problem of long load-times can be reproduced with the Keycloak docker > images and the in-memory database. I also created a small example project > that creates some users with just a few attributes in a docker based 3.4.3 > and 4.3.0 Keycloak environment with a Postgres database to reproduce the > problem. > https://github.com/thomasdarimont/kc-user-regression-tester > > Cheers, > Thomas > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From mposolda at redhat.com Thu Aug 23 04:22:15 2018 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 23 Aug 2018 10:22:15 +0200 Subject: [keycloak-dev] Truststore SPI In-Reply-To: References: Message-ID: Not sure if we should have single SPI or two SPIs? One responsible for CRUD (storage/load/querying) certificates and another for verifying? As it seems that the first will mostly deal with our DB and Infinispan, when the second with the Elytron? Some more inline... On 20/08/18 10:25, Sebastian Laskawiec wrote: > On Tue, Aug 14, 2018 at 11:30 PM Stian Thorgersen > wrote: > >> Was thinking some more about how to share certificate functionality between >> user authentication and client authentication. Potentially also to SAML. >> >> We could add a Truststore SPI. The TruststoreProvider interface would have >> something like the following methods (quick mock stuff): >> > We already have a Truststore SPI [1]. If I remember correctly, it's being > used when Keycloak needs to create an TLS connection to another IDP. > Perhaps we could come up with another name, like CertificateSpi? > > [1] > https://github.com/keycloak/keycloak/blob/08dbdeb57f678646a30a8ef44a76a8bf50f87e59/server-spi-private/src/main/java/org/keycloak/truststore/TruststoreSpi.java > > >> * boolean verifyCert(CertificateMatcher matcher) >> * Set listExpiring() >> > This one should probably take a date as an argument (or maybe there should > be an overloaded method that takes it?). +1 With regards to this, we have a PR that is opened for longer time. The PR adds support for periodically checking the certificates and find those, which are expired OR about to expire in next N days. And then send email to admin about expired certificates: https://github.com/keycloak/keycloak/pull/5121 This PR has some minor issues in it's current form, especially: - It periodically iterates over all realms and clients and retrieves their certificates, which has some performance impact. It also converts every found certificate from String (PEM) to the binary format to check in the certificate itself if it's expired. Converting from PEM to the binary has also some non-trivial performance impact - It checks just some types of certificates (EG. "saml.signing.certificate" and realm RSA certificates) but likely ignore some other types The ability to store some metadata in our DB and being able to effectively query all expired certificates OR those about to expire in next N days, will be very nice to solve both issues. > > >> * void trustCert(Cert cert) >> > Probably addCert would be a better name since we have a removeTrustedCert > method. > > >> * void removeTrustedCert(String id) >> * Cert getSavedCert(String id) >> * void removeSavedCert(String id) >> * void saveCert(Cert cert) >> > Plus, methods that list trusted and saved certs. > > >> We'd have a default built-in TruststoreProvider. Potentially delegating >> some stuff to Elytron, but it would have to store certs in the db. It would >> deal with checking if cert is still valid, call cert validation like the >> user cert does, etc.. >> > That's a very good idea. Some of the interesting stuff is, like > Certificates Revocation Lists are coming out of the box once we integrate > Elytron [2]. > > [2] > https://docs.jboss.org/author/display/WFLY/SSL+Configuration+using+Elytron+Subsystem#SSLConfigurationusingElytronSubsystem-UsingaCertificateRevocationList > > >> For user and client authentication we'd use verifyCert primarily. We'd have >> something like UserCertMatcher.create()... and >> ClientCertMatcher.create()... In the admin console we could add an option >> to add trusted certs (not sure how that'd work in a cluster). > > It shouldn't be hard. We probably need a DB entity for that and let > Infinispan do the caching. Although, I'm not sure if we have any > constraints from the security perspective?? IMO if those will be local caches, the security impact should be fine (we already stored various keys in infinispan in binary format, but just in local caches). Marek > > >> For SAML >> clients we'd delegate storing certs to the Truststore SPI instead of >> storing the cert data alongside the client. The truststore would also add >> some metadata in the db about certs so we can easily query for expiring >> certs. >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From Sebastian.Schuster at bosch-si.com Thu Aug 23 09:19:31 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Thu, 23 Aug 2018 13:19:31 +0000 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: <2254776fb5b84184b8507c0ea24bd14e@bosch-si.com> We are also seeing quite slow response times (6-8 seconds with 90K users) on the user endpoint. We went to 4.3.0.Final directly so I cannot compare to 3.4.3.Final. I do have this monitored in New Relic and it looks like a query called "MSSQL user_attribute select" by New Relic is taking most of the time. New Relic shows the query content as: select attributes0_.USER_ID AS USER_ID4_71_1_, attributes0_.ID AS ID1_71_1_, attributes0_.ID AS ID1_71_0_, attributes0_.NAME AS NAME2_71_0_, attributes0_.USER_ID AS USER_ID4_71_0_, attributes0_.VALUE AS VALUE3_71_0_ FROM USER_ATTRIBUTE attributes0_ WHERE attributes0_.USER_ID IN (select userentity0_.ID FROM USER_ENTITY userentity0_ WHERE userentity0_.REALM_ID=? AND (userentity0_.SERVICE_ACCOUNT_CLIENT_LINK IS ?) ) Does anybody have an idea what this is? Otherwise, I will continue to investigate... Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch?Software Innovations?GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn -----Original Message----- From: keycloak-dev-bounces at lists.jboss.org On Behalf Of Stian Thorgersen Sent: Mittwoch, 22. August 2018 21:25 To: Thomas Darimont Cc: keycloak-dev Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final That's quite worrying. Is it limited to browsing users through the admin console or are you seeing bad behaviour elsewhere? Same question applies to the heap. My best bet here is that it has that something has changed around user querying/caching. On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hello Keycloak team, > > has anyone encountered some performance issues after upgrading 3.4.3 > to 4.x (4.3.0)? > > Today I noticed a performance regression while preparing an upgrade > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. > > In our test environment, we have around ~100k test users stored in a > postgres-backed database. When we started the server with the new > Keycloak version, the migration went through, and everything looked > fine at first, but when we tried to browse the list of users via the > admin-console, we noticed that the CPU and memory consumption of the > server increased significantly, up to a point where Keycloak crashed with an OOME. > > All previous Keycloak versions including 3.4.3 were very modest with > their memory requirements and quite happy with ~1g heap. > However, that seems to have changed in Keycloak 4.3.0 - there we > needed at least 4g to prevent Keycloak from crashing with an OOME. > Furthermore, we noticed that the response times for browsing the > paginated user view increased significantly as well: > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation > takes > ~7 seconds for a test realm with just 10k users. > > In the test realm with 100k users, the time to load a single page in > the users listing was 66 seconds for version 4.3.0, on average - > compared to quite stable 80ms in 3.4.3. > > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 > seconds for 100k users, so I assume the processing logic in Keycloak > is the culprit. > > The problem of long load-times can be reproduced with the Keycloak > docker images and the in-memory database. I also created a small > example project that creates some users with just a few attributes in > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres > database to reproduce the problem. > https://github.com/thomasdarimont/kc-user-regression-tester > > Cheers, > Thomas > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From Sebastian.Schuster at bosch-si.com Thu Aug 23 09:22:42 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Thu, 23 Aug 2018 13:22:42 +0000 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: Btw. we also have 40K user attributes... Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch?Software Innovations?GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn -----Original Message----- From: keycloak-dev-bounces at lists.jboss.org On Behalf Of Stian Thorgersen Sent: Mittwoch, 22. August 2018 21:25 To: Thomas Darimont Cc: keycloak-dev Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final That's quite worrying. Is it limited to browsing users through the admin console or are you seeing bad behaviour elsewhere? Same question applies to the heap. My best bet here is that it has that something has changed around user querying/caching. On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hello Keycloak team, > > has anyone encountered some performance issues after upgrading 3.4.3 > to 4.x (4.3.0)? > > Today I noticed a performance regression while preparing an upgrade > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. > > In our test environment, we have around ~100k test users stored in a > postgres-backed database. When we started the server with the new > Keycloak version, the migration went through, and everything looked > fine at first, but when we tried to browse the list of users via the > admin-console, we noticed that the CPU and memory consumption of the > server increased significantly, up to a point where Keycloak crashed with an OOME. > > All previous Keycloak versions including 3.4.3 were very modest with > their memory requirements and quite happy with ~1g heap. > However, that seems to have changed in Keycloak 4.3.0 - there we > needed at least 4g to prevent Keycloak from crashing with an OOME. > Furthermore, we noticed that the response times for browsing the > paginated user view increased significantly as well: > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation > takes > ~7 seconds for a test realm with just 10k users. > > In the test realm with 100k users, the time to load a single page in > the users listing was 66 seconds for version 4.3.0, on average - > compared to quite stable 80ms in 3.4.3. > > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 > seconds for 100k users, so I assume the processing logic in Keycloak > is the culprit. > > The problem of long load-times can be reproduced with the Keycloak > docker images and the in-memory database. I also created a small > example project that creates some users with just a few attributes in > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres > database to reproduce the problem. > https://github.com/thomasdarimont/kc-user-regression-tester > > Cheers, > Thomas > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From thomas.darimont at googlemail.com Thu Aug 23 09:44:51 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Thu, 23 Aug 2018 15:44:51 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: Hi Sebastian, thanks for investigating! Unfortunately, I didn't have time yet to look further into the problem, but I also observed the slow attributes handling. Btw. we have around 100k users and ~700k attributes. The Entity-Attribute-Value data model takes it's toll here... @Stian so far I've only observed this when scrolling through the user overview in the admin-console, which also triggers the mentioned spike in heap usage. Cheers, Thomas Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1) < Sebastian.Schuster at bosch-si.com>: > Btw. we also have 40K user attributes... > > Best regards, > Sebastian > > Mit freundlichen Gr??en / Best regards > > Dr.-Ing. Sebastian Schuster > > Engineering and Support (INST/ESY1) > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > GERMANY | www.bosch-si.com > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > Sebastian.Schuster at bosch-si.com > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > Stefan Ferber, Michael Hahn > > > > > -----Original Message----- > From: keycloak-dev-bounces at lists.jboss.org < > keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen > Sent: Mittwoch, 22. August 2018 21:25 > To: Thomas Darimont > Cc: keycloak-dev > Subject: Re: [keycloak-dev] Potential performance regression between > 3.4.3.Final and 4.3.0.Final > > That's quite worrying. Is it limited to browsing users through the admin > console or are you seeing bad behaviour elsewhere? Same question applies to > the heap. My best bet here is that it has that something has changed around > user querying/caching. > > On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > > > Hello Keycloak team, > > > > has anyone encountered some performance issues after upgrading 3.4.3 > > to 4.x (4.3.0)? > > > > Today I noticed a performance regression while preparing an upgrade > > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. > > > > In our test environment, we have around ~100k test users stored in a > > postgres-backed database. When we started the server with the new > > Keycloak version, the migration went through, and everything looked > > fine at first, but when we tried to browse the list of users via the > > admin-console, we noticed that the CPU and memory consumption of the > > server increased significantly, up to a point where Keycloak crashed > with an OOME. > > > > All previous Keycloak versions including 3.4.3 were very modest with > > their memory requirements and quite happy with ~1g heap. > > However, that seems to have changed in Keycloak 4.3.0 - there we > > needed at least 4g to prevent Keycloak from crashing with an OOME. > > Furthermore, we noticed that the response times for browsing the > > paginated user view increased significantly as well: > > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In > > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation > > takes > > ~7 seconds for a test realm with just 10k users. > > > > In the test realm with 100k users, the time to load a single page in > > the users listing was 66 seconds for version 4.3.0, on average - > > compared to quite stable 80ms in 3.4.3. > > > > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 > > seconds for 100k users, so I assume the processing logic in Keycloak > > is the culprit. > > > > The problem of long load-times can be reproduced with the Keycloak > > docker images and the in-memory database. I also created a small > > example project that creates some users with just a few attributes in > > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres > > database to reproduce the problem. > > https://github.com/thomasdarimont/kc-user-regression-tester > > > > Cheers, > > Thomas > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From gary.schulte at opengov.com Thu Aug 23 14:54:18 2018 From: gary.schulte at opengov.com (Gary Schulte) Date: Thu, 23 Aug 2018 11:54:18 -0700 Subject: [keycloak-dev] Authenticator SPI: correct way to recover from JPA PersistenceException Message-ID: Keycloak 4.1, standalone-ha, postgres db: As a part of our migration to keycloak, we have a custom authenticator we are using for the direct grant flow which will "lazily create" users. If we fail to find the user in local storage, we create the user with session data from a legacy IDP. This works well, except in the rare case where multiple services are authenticating a user for the first time, we end up with a race condition that yields a ModelDuplicateException: [0m [31m17:31:41,417 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default task-8) Uncaught server error: org.keycloak.models.ModelDuplicateException: javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement at org.keycloak.connections.jpa.PersistenceExceptionConverter.convert(PersistenceExceptionConverter.java:57) at org.keycloak.connections.jpa.PersistenceExceptionConverter.invoke(PersistenceExceptionConverter.java:51) at com.sun.proxy.$Proxy79.flush(Unknown Source) at org.keycloak.models.jpa.JpaUserProvider.addUser(JpaUserProvider.java:94) at com.opengov.auth.keycloak.delphius.DelphiusAuthenticator.authenticate(DelphiusAuthenticator.java:123) at org.keycloak.authentication.DefaultAuthenticationFlow.processFlow(DefaultAuthenticationFlow.java:221) at org.keycloak.authentication.AuthenticationProcessor.authenticateOnly(AuthenticationProcessor.java:910) at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.resourceOwnerPasswordCredentialsGrant(TokenEndpoint.java:547) at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.processGrantRequest(TokenEndpoint.java:184) ... Ideally we could avoid a 500 response and have the race 'loser' look up the newly created user, but the session's transaction gets marked rollback, and further lookup/sql operations fail. I am not finding a way to flush or restart the transaction state in the Provider. I have implemented a failureChallenge that is essentially a 302 redirect back to the same token endpoint, but there are circumstances where a redirect loop occurs. I can sort out and handle these cases, but it feels a bit crufty. Is there a better/prescribed way to recover from JPA session errors in a custom Authenticator? TIA Gary S From johannes at kodet.no Fri Aug 24 04:46:52 2018 From: johannes at kodet.no (Johannes Knutsen) Date: Fri, 24 Aug 2018 10:46:52 +0200 Subject: [keycloak-dev] Locale cookie vs ui_locales priority In-Reply-To: <70b769f9-af6d-ec50-bac7-413d3bfe4994@redhat.com> References: <70b769f9-af6d-ec50-bac7-413d3bfe4994@redhat.com> Message-ID: Thanks for the input! I will create a Jira for the feature and submit a PR after doing some adjustments according to your comments. Not sure about the configuration support. I tend to agree with Stan and prefer to have a sane default without configuration support. Special cases probably will not be supported by configuration alone anyways. At least I think it should be a separate feature request. On Wed, Aug 22, 2018 at 2:06 PM Stan Silvert wrote: > > On 8/22/2018 3:47 AM, Marek Posolda wrote: > > +1 for the LocaleSelectorSPI. Now I recall that on the previous project > > I was working few years ago (GateIn Portal) we had such flexibility due > > the people having different requirements on how the locale should be > > chosen :) > > > > Regarding your proposal, It looks like a good start. But maybe I would > > rather remove the class LocaleHelper entirely and extract the stuff > > directly to the DefaultLocaleSelectorProvider? This will have an > > advantage that DefaultLocaleSelectorProvider may have various protected > > methods (EG. like getLocaleFromCookie, getLocaleFromUiLocalesParameter, > > getLocaleFromUserProfile etc), so that if someone wants to override > > default behaviour, he doesn't need to extract whole code, but just > > extend from DefaultLocaleSelectorProvider and call protected methods > > directly etc? > > > > Also I wonder if DefaultLocaleSelectorProvider should have some > > configuration option, which will allow to easily specify default order > > (EG. "cookie kclocale_param profile ui_locale_param accept_header realm" > > ) so that 90% of people can just configure the value in standalone.xml > > without a need to implement their own LocaleSelectorProvider? > -1 to configuration via standalone.xml. In general, I don't think we > should create a simplified interface to address an edge case. That just > introduces ongoing maintenance. What we have now gets it right 99.9% of > the time. > > +1 to LocaleSelectorSPI. Even if we did the standalone.xml thing, > somebody would still need LocaleSelectorSPI for complex cases. > > > > Marek > > > > On 15/08/18 18:13, Johannes Knutsen wrote: > >> I did some quick work on extracting an LocaleSelectorSPI. Please have > >> a look at https://github.com/keycloak/keycloak/compare/master...knutz3n:feature/locale-spi. > >> Thoughts? > >> > >> - Johannes > >> > >> On Tue, Aug 14, 2018 at 8:32 PM, Stian Thorgersen wrote: > >>> The most common case is that the same user uses the browser. So if the user > >>> has actively changed the locale in Keycloak it should stay like that. I > >>> would actually suggest that if the user has changed the locale in Keycloak, > >>> you should have the locale added to the token so the application can update > >>> it's locale. That would result in all applications in the realm using the > >>> same locale for the user. > >>> > >>> Now if we change it to what you are suggesting you can end up with > >>> situations where the first login page is displayed in one language, then > >>> once the user has entered username/email it would end up switching to the > >>> locale associated with the user. > >>> > >>> If you really don't like that behaviour I would suggest we either introduce > >>> a LocaleSelectorSPI where you can change the logic with a custom provider. > >>> Alternatively, we could potentially add an option on the realm to be able to > >>> disable the locale selector on Keycloak, which means that the user wouldn't > >>> be able to change the locale and ui_locales would always take precedence. In > >>> either case I think it's best to keep the current behaviour as the default, > >>> with some way of being able to change it. > >>> > >>> On Tue, 14 Aug 2018 at 12:56, Johannes Knutsen wrote: > >>>> It currently works the way you describe it. The ui_locales parameter > >>>> is taking effect as long as the locale cookie isn't set. However, the > >>>> logic implies that you always expect the same user to authenticate and > >>>> our preference is that the user should get a login page matching the > >>>> locale of the client which sends the user to the login page, if the > >>>> client specified preferred locales as an ui_locales parameter. > >>>> Using the locale cookie to override the ui_locales parameter makes the > >>>> user experience inconsistent when moving from a client with a locale > >>>> different from the last logged in user's preferred locale. > >>>> > >>>> A proposed change is to change locale priority as follows: > >>>> 1. kc_locale query parameter. This updates the user's locale and locale > >>>> cookie. > >>>> 2. user profile locale if a user is authenticated. This updates the > >>>> locale cookie. > >>>> 3. ui_locales query parameter. If ui_locales contains the locale > >>>> cookie value, return it. Otherwise, return the first locale available > >>>> from ui_locales > >>>> 4. locale cookie > >>>> 5. accept language http header > >>>> > >>>> Is this change something you would consider merging if I create a pull > >>>> request? Any other adjustments or thoughts regarding such a change? > >>>> > >>>> - Johannes > >>>> > >>>> On Tue, Aug 14, 2018 at 9:16 AM, Stian Thorgersen > >>>> wrote: > >>>>> The way at least it was supposed to work was that the Locale cookie > >>>>> would > >>>>> only be set if a user has explicitly changed the locale. That way we can > >>>>> remember a users preference even when the user is logged out. > >>>>> > >>>>> Are you seeing ui_locales not taking affect without a user having used > >>>>> the > >>>>> locale drop-down? > >>>>> > >>>>> On Mon, 13 Aug 2018 at 23:42, Johannes Knutsen > >>>>> wrote: > >>>>>> Currently the LocaleHelper#getUserLocale chooses locale priority based > >>>>>> on the following order: > >>>>>> > >>>>>> 1. kc_locale query parameter > >>>>>> 2. Locale cookie > >>>>>> 3. User profile > >>>>>> 4. ui_locales query parameter > >>>>>> 5. Accept-Language http header > >>>>>> > >>>>>> We are building clients which uses the ui_locales parameter to select > >>>>>> localization of the login to match the locale of the client. > >>>>>> Since the locale cookie has a higher priority than the ui_locales > >>>>>> query parameter, the ui_locales query parameter is ignored for any > >>>>>> auth requests with a kc_locale cookie present. > >>>>>> > >>>>>> 1. Do you see any implications of prioritizing the ui_locales > >>>>>> parameter above the locale cookie when resolving locale? > >>>>>> 2. Are there reasons why the ui_locales does not update the locale > >>>>>> cookie value the same way kc_locale parameter does it? > >>>>>> > >>>>>> Best regards, > >>>>>> Johannes Knutsen > >>>>>> _______________________________________________ > >>>>>> keycloak-dev mailing list > >>>>>> keycloak-dev at lists.jboss.org > >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From jambo_mcd at yahoo.co.uk Fri Aug 24 05:15:07 2018 From: jambo_mcd at yahoo.co.uk (Jamie McDowell) Date: Fri, 24 Aug 2018 09:15:07 +0000 (UTC) Subject: [keycloak-dev] Keycloak to OpenLDAP - Password Policies References: <122960986.2842630.1535102107930.ref@mail.yahoo.com> Message-ID: <122960986.2842630.1535102107930@mail.yahoo.com> Hi Dev's, Appreciate if you can help me with an issue i have with password policies federating from Keycloak (v3.4.3) to OpenLDAP. I have created 2 password policies on the OpenLDAP server, where i require one for end users and one for service accounts - these are defined in the specific OU's to where the accounts are held.? I have set the password policies for both users and service accounts (policy module, schema, overlay etc..) and can confirm that the policy is being picked up on the OpenLDAP host when i run the command ldappasswd for the user and enter less characters than the required password length (for example) The issue i have is that within keycloak i haven't set any password policies as i would like this to use the one i have created within the OpenLDAP server. Can Keycloak be configured that this must check against the OpenLDAP password policy? I have one realm set up along with a client. I have been trying to get this working now for the last 10 days and not getting very far.? Within my LDAP Mapper i have tried creating a msad-user-account-control-mapper however this does not work, i get provided with an error when resetting my user password "Failed to update password in Active Directory. Exception message: [LDAP: error code 17 - pwdLastSet: attribute type undefined" I would have expected something like this considering i am not using AD.? Any suggestions would be appreciated? Regards,Jamie From Mariusz.Zawadzki at tomtom.com Fri Aug 24 06:10:57 2018 From: Mariusz.Zawadzki at tomtom.com (Mariusz Zawadzki) Date: Fri, 24 Aug 2018 10:10:57 +0000 Subject: [keycloak-dev] KEYCLOAK-4593 - large number of realms - approaching a solution Message-ID: Hi. We're considering Keycloak as a solution for us to Authenticate and Authorize users. We want to have separate realm for a customer but that requires hundreds possibly more realms in one keycloak. I've looked into the code and currently the main bottle neck I see are calls to: session.realms().getRealms(); During application initialization. I've got a simple solution to one of those usages, so my question is. What is the general approach to do this? On github it is "one commit per issue", but the whole problem with many realms looks to have a lot of places to improve, and currently we are blocked by two places: org.keycloak.services.managers.ApplianceBootstrap#isNewInstall org.keycloak.services.managers.UserStorageSyncManager#bootstrapPeriodic Regards, Mariusz Zawadzki From johannes at kodet.no Sun Aug 26 16:28:35 2018 From: johannes at kodet.no (Johannes Knutsen) Date: Sun, 26 Aug 2018 22:28:35 +0200 Subject: [keycloak-dev] Allow passing current locale to OAuth2 IdPs Message-ID: We would like to have the user's current locale passed along to an OAuth2 identity provider as a ui_locales parameter. This would allow the user to select a custom locale on our Keycloak instance and continue to have same locale on the external IdP. Any thoughts on such a feature? I have created an example implementation, see https://github.com/keycloak/keycloak/compare/master...knutz3n:feature/pass_current_locale_to_idp, and I will add tests and create a PR if you would like to merge it. Regards, Johannes Knutsen From Sebastian.Schuster at bosch-si.com Mon Aug 27 07:26:46 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Mon, 27 Aug 2018 11:26:46 +0000 Subject: [keycloak-dev] Possible feature: role attributes Message-ID: <10b624a57cea40a1bcc9bd77b63b398a@bosch-si.com> Hi everybody, We have a use case where we would like to store additional meta-information for roles. This come from our IAM-requirements, that say there is a single responsible person for a role or that roles give access to data with different classifications. One way to store this kind of information would be to introduce role attributes to client and realm roles, basically similar to user or group attributes. For us, it would be sufficient to have this information purely as metadata, i.e. we would only read it through the audit log to inform the responsible person about role assignments if a role with a certain classification is assigned. In contrast to that, you can add group und user attributes to a token using user attribute mappers and the client application can extract this information from the token and act on it. WDYT? Does anybody else have similar requirements? Would you need role custom attributes also in the token? I can imagine that it gets kind of difficult to identify where attributes come from, once there are user, group, and role attributes, possibly with inheritance/composition. Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn From sthorger at redhat.com Mon Aug 27 07:33:43 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 27 Aug 2018 13:33:43 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: Can you open a bug for this in JIRA please and include as much details as you can. On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hi Sebastian, > > thanks for investigating! Unfortunately, I didn't have time yet to look > further into the problem, but I also observed the slow attributes handling. > Btw. we have around 100k users and ~700k attributes. The > Entity-Attribute-Value data model takes it's toll here... > > @Stian so far I've only observed this when scrolling through the user > overview in the admin-console, which also triggers > the mentioned spike in heap usage. > > Cheers, > Thomas > > Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com>: > >> Btw. we also have 40K user attributes... >> >> Best regards, >> Sebastian >> >> Mit freundlichen Gr??en / Best regards >> >> Dr.-Ing. Sebastian Schuster >> >> Engineering and Support (INST/ESY1) >> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> GERMANY | www.bosch-si.com >> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> Sebastian.Schuster at bosch-si.com >> >> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. >> Stefan Ferber, Michael Hahn >> >> >> >> >> -----Original Message----- >> From: keycloak-dev-bounces at lists.jboss.org < >> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen >> Sent: Mittwoch, 22. August 2018 21:25 >> To: Thomas Darimont >> Cc: keycloak-dev >> Subject: Re: [keycloak-dev] Potential performance regression between >> 3.4.3.Final and 4.3.0.Final >> >> That's quite worrying. Is it limited to browsing users through the admin >> console or are you seeing bad behaviour elsewhere? Same question applies to >> the heap. My best bet here is that it has that something has changed around >> user querying/caching. >> >> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < >> thomas.darimont at googlemail.com> wrote: >> >> > Hello Keycloak team, >> > >> > has anyone encountered some performance issues after upgrading 3.4.3 >> > to 4.x (4.3.0)? >> > >> > Today I noticed a performance regression while preparing an upgrade >> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. >> > >> > In our test environment, we have around ~100k test users stored in a >> > postgres-backed database. When we started the server with the new >> > Keycloak version, the migration went through, and everything looked >> > fine at first, but when we tried to browse the list of users via the >> > admin-console, we noticed that the CPU and memory consumption of the >> > server increased significantly, up to a point where Keycloak crashed >> with an OOME. >> > >> > All previous Keycloak versions including 3.4.3 were very modest with >> > their memory requirements and quite happy with ~1g heap. >> > However, that seems to have changed in Keycloak 4.3.0 - there we >> > needed at least 4g to prevent Keycloak from crashing with an OOME. >> > Furthermore, we noticed that the response times for browsing the >> > paginated user view increased significantly as well: >> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In >> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation >> > takes >> > ~7 seconds for a test realm with just 10k users. >> > >> > In the test realm with 100k users, the time to load a single page in >> > the users listing was 66 seconds for version 4.3.0, on average - >> > compared to quite stable 80ms in 3.4.3. >> > >> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 >> > seconds for 100k users, so I assume the processing logic in Keycloak >> > is the culprit. >> > >> > The problem of long load-times can be reproduced with the Keycloak >> > docker images and the in-memory database. I also created a small >> > example project that creates some users with just a few attributes in >> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres >> > database to reproduce the problem. >> > https://github.com/thomasdarimont/kc-user-regression-tester >> > >> > Cheers, >> > Thomas >> > _______________________________________________ >> > keycloak-dev mailing list >> > keycloak-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > From sthorger at redhat.com Mon Aug 27 07:35:00 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 27 Aug 2018 13:35:00 +0200 Subject: [keycloak-dev] Allow passing current locale to OAuth2 IdPs In-Reply-To: References: Message-ID: Sounds good. I had a quick look at your example implementation and that looks like the correct approach to me. On Sun, 26 Aug 2018 at 23:38, Johannes Knutsen wrote: > We would like to have the user's current locale passed along to an > OAuth2 identity provider as a ui_locales parameter. > This would allow the user to select a custom locale on our Keycloak > instance and continue to have same locale on the external IdP. > > Any thoughts on such a feature? > > I have created an example implementation, see > > https://github.com/keycloak/keycloak/compare/master...knutz3n:feature/pass_current_locale_to_idp > , > and I will add tests and create a PR if you would like to merge it. > > Regards, > Johannes Knutsen > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Mon Aug 27 07:36:56 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 27 Aug 2018 13:36:56 +0200 Subject: [keycloak-dev] KEYCLOAK-4593 - large number of realms - approaching a solution In-Reply-To: References: Message-ID: Ideal would be to break the problem into smaller chunks. So create a JIRA for problematic area and send a PR for it. You can link it to the support many realms JIRA. On Fri, 24 Aug 2018 at 12:18, Mariusz Zawadzki wrote: > Hi. > > We're considering Keycloak as a solution for us to Authenticate and > Authorize users. > We want to have separate realm for a customer but that requires hundreds > possibly more realms in one keycloak. > > I've looked into the code and currently the main bottle neck I see are > calls to: > session.realms().getRealms(); > > During application initialization. > > I've got a simple solution to one of those usages, so my question is. > > What is the general approach to do this? > On github it is "one commit per issue", but the whole problem with many > realms looks to have a lot of places to improve, and currently we are > blocked by two places: > > org.keycloak.services.managers.ApplianceBootstrap#isNewInstall > org.keycloak.services.managers.UserStorageSyncManager#bootstrapPeriodic > > > Regards, > Mariusz Zawadzki > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Mon Aug 27 07:49:23 2018 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 27 Aug 2018 13:49:23 +0200 Subject: [keycloak-dev] Possible feature: role attributes In-Reply-To: <10b624a57cea40a1bcc9bd77b63b398a@bosch-si.com> References: <10b624a57cea40a1bcc9bd77b63b398a@bosch-si.com> Message-ID: I don't think we need to consider adding role attributes to the token. That would very quickly bloat tokens. I would like to see a bit more general use of role attributes as part of incorporating such a feature. Otherwise it would end up being a rather hidden feature. Some ideas I have in mind: * Ability to do crud of role attributes in admin console * Ability to query for roles based on attributes For future work it would be great to have attributes on everything. That would allow us to do something like OpenShift `oc` does. Where you're able to search and delete everything based on attributes. One nice use-case here is that you can tag all clients, roles, etc.. that belong to a deployment (a group of apps and services) and be able to view everything that is related to the deployment in Keycloak. On Mon, 27 Aug 2018 at 13:32, Schuster Sebastian (INST/ESY1) < Sebastian.Schuster at bosch-si.com> wrote: > Hi everybody, > > We have a use case where we would like to store additional > meta-information for roles. This come from our IAM-requirements, that say > there is a single responsible person for a role or that roles give access > to data with different classifications. One way to store this kind of > information would be to introduce role attributes to client and realm > roles, basically similar to user or group attributes. > > For us, it would be sufficient to have this information purely as > metadata, i.e. we would only read it through the audit log to inform the > responsible person about role assignments if a role with a certain > classification is assigned. In contrast to that, you can add group und user > attributes to a token using user attribute mappers and the client > application can extract this information from the token and act on it. > > WDYT? Does anybody else have similar requirements? Would you need role > custom attributes also in the token? I can imagine that it gets kind of > difficult to identify where attributes come from, once there are user, > group, and role attributes, possibly with inheritance/composition. > > Best regards, > Sebastian > > Mit freundlichen Gr??en / Best regards > > Dr.-Ing. Sebastian Schuster > > Engineering and Support (INST/ESY1) > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > GERMANY | www.bosch-si.com > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > Sebastian.Schuster at bosch-si.com > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > Stefan Ferber, Michael Hahn > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From thomas.darimont at googlemail.com Mon Aug 27 08:37:59 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Mon, 27 Aug 2018 14:37:59 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: Hello, FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 Cheers, Thomas Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < sthorger at redhat.com>: > Can you open a bug for this in JIRA please and include as much details as > you can. > > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > >> Hi Sebastian, >> >> thanks for investigating! Unfortunately, I didn't have time yet to look >> further into the problem, but I also observed the slow attributes handling. >> Btw. we have around 100k users and ~700k attributes. The >> Entity-Attribute-Value data model takes it's toll here... >> >> @Stian so far I've only observed this when scrolling through the user >> overview in the admin-console, which also triggers >> the mentioned spike in heap usage. >> >> Cheers, >> Thomas >> >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1) >> : >> >>> Btw. we also have 40K user attributes... >>> >>> Best regards, >>> Sebastian >>> >>> Mit freundlichen Gr??en / Best regards >>> >>> Dr.-Ing. Sebastian Schuster >>> >>> Engineering and Support (INST/ESY1) >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >>> GERMANY | www.bosch-si.com >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >>> Sebastian.Schuster at bosch-si.com >>> >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >>> Dr. Stefan Ferber, Michael Hahn >>> >>> >>> >>> >>> -----Original Message----- >>> From: keycloak-dev-bounces at lists.jboss.org < >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen >>> Sent: Mittwoch, 22. August 2018 21:25 >>> To: Thomas Darimont >>> Cc: keycloak-dev >>> Subject: Re: [keycloak-dev] Potential performance regression between >>> 3.4.3.Final and 4.3.0.Final >>> >>> That's quite worrying. Is it limited to browsing users through the admin >>> console or are you seeing bad behaviour elsewhere? Same question applies to >>> the heap. My best bet here is that it has that something has changed around >>> user querying/caching. >>> >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < >>> thomas.darimont at googlemail.com> wrote: >>> >>> > Hello Keycloak team, >>> > >>> > has anyone encountered some performance issues after upgrading 3.4.3 >>> > to 4.x (4.3.0)? >>> > >>> > Today I noticed a performance regression while preparing an upgrade >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. >>> > >>> > In our test environment, we have around ~100k test users stored in a >>> > postgres-backed database. When we started the server with the new >>> > Keycloak version, the migration went through, and everything looked >>> > fine at first, but when we tried to browse the list of users via the >>> > admin-console, we noticed that the CPU and memory consumption of the >>> > server increased significantly, up to a point where Keycloak crashed >>> with an OOME. >>> > >>> > All previous Keycloak versions including 3.4.3 were very modest with >>> > their memory requirements and quite happy with ~1g heap. >>> > However, that seems to have changed in Keycloak 4.3.0 - there we >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. >>> > Furthermore, we noticed that the response times for browsing the >>> > paginated user view increased significantly as well: >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation >>> > takes >>> > ~7 seconds for a test realm with just 10k users. >>> > >>> > In the test realm with 100k users, the time to load a single page in >>> > the users listing was 66 seconds for version 4.3.0, on average - >>> > compared to quite stable 80ms in 3.4.3. >>> > >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 >>> > seconds for 100k users, so I assume the processing logic in Keycloak >>> > is the culprit. >>> > >>> > The problem of long load-times can be reproduced with the Keycloak >>> > docker images and the in-memory database. I also created a small >>> > example project that creates some users with just a few attributes in >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres >>> > database to reproduce the problem. >>> > https://github.com/thomasdarimont/kc-user-regression-tester >>> > >>> > Cheers, >>> > Thomas >>> > _______________________________________________ >>> > keycloak-dev mailing list >>> > keycloak-dev at lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> From gary.schulte at opengov.com Mon Aug 27 13:30:10 2018 From: gary.schulte at opengov.com (Gary Schulte) Date: Mon, 27 Aug 2018 10:30:10 -0700 Subject: [keycloak-dev] UMA2.0 & Policy permission evaluations for RPTs Message-ID: I encountered this late last week and created a JIRA for it, but in retrospect I should probably have brought it up on the list as well. https://issues.jboss.org/browse/KEYCLOAK-8134 briefly, for a uma 2.0 managed realm, I am seeing inconsistent behavior when getting an RPT. When I request an RPT for the uma grant type (urn:ietf:params:oauth:grant-type:uma-ticket) the policy/permissions are not evaluated unless I specify some combination of resources/scopes for the permission parameter(s). I was expecting an unfiltered RPT to come back with permissions that are specifically granted by policy as well as those granted by UMA2. As it is, I have worked around it by specifying all of the "scope permissions's" scopes (without resources) in the permission params. e.g. ...&permission=#edit&permission=#view&permission=#owner I am encountering this on 4.1.0.Final and it appears to be present in latest (4.3.0.Final) Is this expected behavior? -- Gary Schulte I Software Engineer OpenGov 505-750-4279 gary.schulte at opengov.com www.opengov.com Silicon Valley | Washington DC From adesbiaux at vente-privee.com Tue Aug 28 05:39:41 2018 From: adesbiaux at vente-privee.com (Adrien DESBIAUX) Date: Tue, 28 Aug 2018 09:39:41 +0000 Subject: [keycloak-dev] Infinispan Cache and Keycloak instance restart Message-ID: Hello! Thanks again for maintaining Keycloak. I would have a question: - We are running Keycloak 3.4 - We did setup a Keycloak standalone HA cluster (3 nodes). - As well we did connect those nodes to an Infinispan cluster. All is running fine. However the trouble comes when we want to update the Keycloak Nodes. Updates can be either on themes or modules. We update for example a module, e.g a new user federation SPI, with Ansible. Then we restart the Keycloak nodes one after another. The restart is required so that a Keycloak node can register the new module. All is correct except that users need to login again after the reload has been performed. Is it something we can prevent happening? it is a strange behaviour, since we rely on Infinispan for sessions cache... Is there another local cache to be configured in Infinispan to avoid users to re-login after restarting Keycloak? It would be very helpful if you could give us a direction to investigate as rolling out several updates per day would lead to several re-logins from a user perspective. Cheers, From psilva at redhat.com Tue Aug 28 07:50:56 2018 From: psilva at redhat.com (Pedro Igor Silva) Date: Tue, 28 Aug 2018 08:50:56 -0300 Subject: [keycloak-dev] UMA2.0 & Policy permission evaluations for RPTs In-Reply-To: References: Message-ID: Hi Gary, This is not the expected behavior. When obtaining all permissions (no permission/ticket parameter set) the server will match resources accordingly to: * Resources where the owner is the *resource server* itself * Resources where the owner is the *user* represented by the access/ID token you sent as a bearer to the token endpoint * Resources where the *user* was granted with access through async authorization based on UMA flow (owner have approved access via account service, for instance) Does the resources you are expecting match any of these conditions ? Regards. Pedro Igor On Mon, Aug 27, 2018 at 2:30 PM, Gary Schulte wrote: > I encountered this late last week and created a JIRA for it, but in > retrospect I should probably have brought it up on the list as well. > > https://issues.jboss.org/browse/KEYCLOAK-8134 > > briefly, for a uma 2.0 managed realm, I am seeing inconsistent behavior > when getting an RPT. When I request an RPT for the uma grant type > (urn:ietf:params:oauth:grant-type:uma-ticket) the policy/permissions are > not evaluated unless I specify some combination of resources/scopes for the > permission parameter(s). > > I was expecting an unfiltered RPT to come back with permissions that are > specifically granted by policy as well as those granted by UMA2. As it is, > I have worked around it by specifying all of the "scope permissions's" > scopes (without resources) in the permission params. e.g. > > ...&permission=#edit&permission=#view&permission=#owner > > I am encountering this on 4.1.0.Final and it appears to be present in > latest (4.3.0.Final) > > Is this expected behavior? > > > -- > > Gary Schulte I Software Engineer > > OpenGov > > 505-750-4279 > > gary.schulte at opengov.com > > www.opengov.com > > Silicon Valley > -122.2121292,15z/data=!4m2!3m1!1s0x0:0xb84d4c3f06ecd893> > | Washington DC > +Washington,+DC+20009/@38.915617,-77.0474907,17z/data=!3m1!4b1!4m2!3m1! > 1s0x89b7b7cf85e25661:0x932fc62149d9247f> > > +Washington,+DC+20009/@38.915617,-77.0474907,17z/data=!3m1!4b1!4m2!3m1! > 1s0x89b7b7cf85e25661:0x932fc62149d9247f> > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From adesbiaux at vente-privee.com Tue Aug 28 08:40:54 2018 From: adesbiaux at vente-privee.com (Adrien DESBIAUX) Date: Tue, 28 Aug 2018 12:40:54 +0000 Subject: [keycloak-dev] SAML client integration HTTP 405 error Message-ID: Hi there, We are running Keycloak 3.4, and we noticed that for some SAML clients, we get a HTTP 405 error. This happen for example with client having a local login and a button to login with SSO. For example Appdynamics does that (to name only that one). They perform a POST request to the Keycloak SAML endpoint of the configured client, leading to a HTTP 405 error. The fact that they perform a POST to load the Keycloak login page is discussable, but how to counter this behaviour on Keycloak client's configuration side? Thank you in advance for your guidance. Cheers, From alistair.doswald at elca.ch Wed Aug 29 03:08:56 2018 From: alistair.doswald at elca.ch (Doswald Alistair) Date: Wed, 29 Aug 2018 07:08:56 +0000 Subject: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project In-Reply-To: References: <4dd6baaaf6654cb79441536c2118f798@elca.ch> Message-ID: Hello, I meant to send you this mail last week, but somehow I left it in my drafts. For the keycloak export module, we have not added any special mechanisms to the standard full export, so all the problems you mention are valid. However, in our use case we run keycloak with high availability, and there?s no possibility to shut keycloak completely down for the current full export system. However, there are some benefits to having a full export ,which we require. for example: * It allows us to have a simple logical export of the database. This export can be used to quickly set up a test system. It is also easier to transform via scripts if we have some more complex changes to perform upon the data than a standard database export (in which case we would export, run scripts and reimport). * It allows us to get the complete persistent state for any of our clients which have problems, without having to plan a maintenance shutdown > * What happens if there are changes while doing the export, we where thinking we'd have to pause the server from receiving requests during this export From our point of view, doing a full export or a partial export while keycloak is running amounts to the same thing. In both cases there?s a risk of inconstancies during the export, though there?s definitely more chance of inconstancies with the full export. Therefore, we shift the difficulty to our import process, which must be able to detect such inconsistencies. Sections with problems can be dropped or corrected manually/via script. > * We don't want a REST endpoint that can return credentials and other secrets. Then there's also the size of the export. It would have to store the export on disc, but then what about clustering. One potential would be to develop a tool that can be executed from anywhere that has access to the db. It would use Keycloak code base, but instead of starting a server it would allow things like running an export from the db, migrate the db, etc I can certainly understand why you don?t want an endpoint that can return secrets, however by restricting to admin access and with https communication it?s not much different than viewing the information in the admin GUI. The size of the export is however a concern, and as you say, saving it on a disk is not practical. In our architecture, we have an interface in front of keycloak through which all API request and responses must pass. We can have this interface provide the necessary buffering/streaming mechanisms in order to transmit the large data to the requestor. Cheers, Alistair From: Stian Thorgersen Sent: jeudi 16 ao?t 2018 10:30 To: Doswald Alistair Cc: keycloak-dev Subject: Re: [keycloak-dev] Keycloak Modules developed for the Cloudtrust project On Tue, 14 Aug 2018 at 12:03, Doswald Alistair > wrote: Hello, I just wanted to let this mailing list know that for the Cloudtrust project (https://github.com/cloudtrust), we have developed a certain number modules for Keycloak. These are currently compatible with the version 3.4.3.Final of Keycloak, but we will make them compatible with Keycloak 4.X (where X will be the latest sub-version of Keycloak when we start working on this) as soon as we can. These modules are: * keycloak-wsfed (https://github.com/cloudtrust/keycloak-wsfed): an implementation of the WS-Federation protocol for keycloak. This allows to select the WS-Federation protocol for Keycloak clients and for identity brokers. This is great and pleased that you are maintaining this. We simply don't have the capacity to have this incorporated into Keycloak as we lack expertise on WS-Fed and have to many other things to maintain. * keycloak-authorization (https://github.com/cloudtrust/keycloak-authorization): this module allows the use of the client authorization system to prevent a user which is authenticated in a Keycloak realm to access a given client. It works no matter which protocol is used, and without the client having to support any extra protocol. Note: this solution is a bit hacky, but necessary for one of our use-cases. Sounds interesting. Can you write some more details about this? In the past I've considered if we should have something like required roles to use to a client. That would by not allowing a user to authenticate with a client, but also could prevent adding audience for a service the user isn't allowed to access. Could perhaps take it one step further and prevent client scopes that users are not allowed to access. * keycloak-client-mappers (https://github.com/cloudtrust/keycloak-client-mappers): a module for adding any mappers that we might need that are not yet part of Keycloak. Currently only contains a JavaScript mapper for SAML, analogous to the OIDC script mapper. I've noticed that there's an open issue for this feature (https://issues.jboss.org/browse/KEYCLOAK-5520). If desirable I could submit this code not as a module but a solution to the issue. SAML JavaScript protocol mapper would be great. If you submit with tests and docs we'd love to have this in. * keycloak-export (https://github.com/cloudtrust/keycloak-export): a module adding an endpoint to fully export a realm while Keycloak is still running (no need for restarts!). Can you write some more details about this? In the past we've considered this, but not added it mainly for a few reasons, including: * What happens if there are changes while doing the export, we where thinking we'd have to pause the server from receiving requests during this export * We don't want a REST endpoint that can return credentials and other secrets. Then there's also the size of the export. It would have to store the export on disc, but then what about clustering. One potential would be to develop a tool that can be executed from anywhere that has access to the db. It would use Keycloak code base, but instead of starting a server it would allow things like running an export from the db, migrate the db, etc. Cheers, Alistair PS: I mailed this to both dev and user mailing lists as I believe it may interest members of both mailing lists. However, upon sending to the dev mailing list the first time it bounced. This is the second attempt. _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From Leon.Graser at bosch-si.com Wed Aug 29 07:18:40 2018 From: Leon.Graser at bosch-si.com (Graser Leon (INST-CSS/BSV-OS)) Date: Wed, 29 Aug 2018 11:18:40 +0000 Subject: [keycloak-dev] Java Admin Client does not return to caller Message-ID: Hi all, We import some data to Keycloak programmatically using the java admin client. Thereby, we experience some problems. Some of the functions to call do not return to the caller. Tracking down for an example brought us to the RolePoliciesResource class which we use in the version of 4.3.0.Final (same as our Keycloak deployment). https://github.com/keycloak/keycloak/blob/4.3.0.Final/integration/admin-client/src/main/java/org/keycloak/admin/client/resource/RolePoliciesResource.java In a simple scenario we simply tried to create a 100 policies and find them afterwards using the two provided functions (create and findByName). This works perfectly fine for exactly 50 policies and then you don't get a return with the 50th call (starting from 0). Has anybody experienced issues with running into loops using the java admin client or knows a resolution to this problem? Mit freundlichen Gr??en / Best regards Leon Graser INST-CSS/BSV-OS Tel. +49 30 726112-284 | Mobil +49 1520 9198324 From Sebastian.Loesch at governikus.de Wed Aug 29 07:20:01 2018 From: Sebastian.Loesch at governikus.de (=?iso-8859-1?Q?L=F6sch=2C_Sebastian?=) Date: Wed, 29 Aug 2018 11:20:01 +0000 Subject: [keycloak-dev] SCIM v2 support Message-ID: <93f62b8abc994e88835f314118fb9f4d@BOSKGEXC01.boskg.local> Hello, in a customer project we use keycloak and need a SCIM (System for Cross-domain Identity Management) API. Currently we write a wrapper API and a custom endpoint providing the SCIM functionality. We wrote a extension of the UserEntity, UserModel and an extension of the JpaUserProvider. This strategy seems not ideal and the nicest way is to add this extensions to Keycloak. This is already suggested in https://issues.jboss.org/browse/KEYCLOAK-2537 Is anybody out there who can guide me, what coding would be necessary to contribute the SCIM functionality? Currently I think we have to: - extend the UserEntity with all SCIM attributes. This will result in additional tables/entities for complex attributes e.g. Address, Name, Email - extend the UserModel to povide the additional attributes - implement the new SCIM endpoint /Users - make the additional attributes available via Admin REST API /users - extend views to be able to edit SCIM user attributes using the web ui - - All the above again for the Groups endpoint This also seem to be major changes. To big for one Pull Request. How do you like to handle this? Best regards, Sebastian From thomas.darimont at googlemail.com Wed Aug 29 08:04:28 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Wed, 29 Aug 2018 14:04:28 +0200 Subject: [keycloak-dev] SCIM v2 support In-Reply-To: <93f62b8abc994e88835f314118fb9f4d@BOSKGEXC01.boskg.local> References: <93f62b8abc994e88835f314118fb9f4d@BOSKGEXC01.boskg.local> Message-ID: Hello Sebastian, having scim support built-in keycloak would be awesome! I'd take a look at other implementations, e.g. the scim support in Cloud Foundys UAA: https://github.com/cloudfoundry/uaa/tree/develop/server/src/main/java/org/cloudfoundry/identity/uaa/scim There you can see what's involved. Cheers, Thomas L?sch, Sebastian schrieb am Mi., 29. Aug. 2018, 13:28: > Hello, > > > > in a customer project we use keycloak and need a SCIM (System for > Cross-domain Identity Management) API. > > Currently we write a wrapper API and a custom endpoint providing the SCIM > functionality. We wrote a extension of the UserEntity, UserModel and an > extension of the JpaUserProvider. > > This strategy seems not ideal and the nicest way is to add this extensions > to Keycloak. This is already suggested in > https://issues.jboss.org/browse/KEYCLOAK-2537 > > Is anybody out there who can guide me, what coding would be necessary to > contribute the SCIM functionality? > > > > Currently I think we have to: > > - extend the UserEntity with all SCIM attributes. This will result > in additional tables/entities for complex attributes e.g. Address, Name, > Email > > - extend the UserModel to povide the additional attributes > > - implement the new SCIM endpoint /Users > > - make the additional attributes available via Admin REST API > /users > > - extend views to be able to edit SCIM user attributes using the > web ui > > - ? > > - All the above again for the Groups endpoint? > > > > This also seem to be major changes. To big for one Pull Request. How do you > like to handle this? > > > > Best regards, > > Sebastian > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From psilva at redhat.com Wed Aug 29 08:12:31 2018 From: psilva at redhat.com (Pedro Igor Silva) Date: Wed, 29 Aug 2018 09:12:31 -0300 Subject: [keycloak-dev] Java Admin Client does not return to caller In-Reply-To: References: Message-ID: Hi, Are you calling close method on Response object ? Regards. Pedro Igor On Wed, Aug 29, 2018 at 8:18 AM, Graser Leon (INST-CSS/BSV-OS) < Leon.Graser at bosch-si.com> wrote: > Hi all, > > We import some data to Keycloak programmatically using the java admin > client. Thereby, we experience some problems. Some of the functions to call > do not return to the caller. Tracking down for an example brought us to the > RolePoliciesResource class which we use in the version of 4.3.0.Final (same > as our Keycloak deployment). > https://github.com/keycloak/keycloak/blob/4.3.0.Final/ > integration/admin-client/src/main/java/org/keycloak/admin/client/resource/ > RolePoliciesResource.java > > In a simple scenario we simply tried to create a 100 policies and find > them afterwards using the two provided functions (create and findByName). > This works perfectly fine for exactly 50 policies and then you don't get a > return with the 50th call (starting from 0). > > Has anybody experienced issues with running into loops using the java > admin client or knows a resolution to this problem? > > > Mit freundlichen Gr??en / Best regards > > Leon Graser > INST-CSS/BSV-OS > > Tel. +49 30 726112-284 | Mobil +49 1520 9198324 > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From hmlnarik at redhat.com Wed Aug 29 10:35:09 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Wed, 29 Aug 2018 16:35:09 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: Thanks Thomas and all, great help with reproducing the issue at least for admin user list endpoint. Could you please test [1] that it indeed fixes the issue? It was caused obtaining too much user details when unnecessary (for user list, only few fields are necessary). As for increased memory requirements, these might be unrelated to this fix, and we'll keep an eye on this. --Hynek [1] https://github.com/keycloak/keycloak/pull/5517 On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hello, > > FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 > > Cheers, > Thomas > > Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < > sthorger at redhat.com>: > > > Can you open a bug for this in JIRA please and include as much details as > > you can. > > > > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < > > thomas.darimont at googlemail.com> wrote: > > > >> Hi Sebastian, > >> > >> thanks for investigating! Unfortunately, I didn't have time yet to look > >> further into the problem, but I also observed the slow attributes > handling. > >> Btw. we have around 100k users and ~700k attributes. The > >> Entity-Attribute-Value data model takes it's toll here... > >> > >> @Stian so far I've only observed this when scrolling through the user > >> overview in the admin-console, which also triggers > >> the mentioned spike in heap usage. > >> > >> Cheers, > >> Thomas > >> > >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian > (INST/ESY1) > >> : > >> > >>> Btw. we also have 40K user attributes... > >>> > >>> Best regards, > >>> Sebastian > >>> > >>> Mit freundlichen Gr??en / Best regards > >>> > >>> Dr.-Ing. Sebastian Schuster > >>> > >>> Engineering and Support (INST/ESY1) > >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > >>> GERMANY | www.bosch-si.com > >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > >>> Sebastian.Schuster at bosch-si.com > >>> > >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: > >>> Dr. Stefan Ferber, Michael Hahn > >>> > >>> > >>> > >>> > >>> -----Original Message----- > >>> From: keycloak-dev-bounces at lists.jboss.org < > >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen > >>> Sent: Mittwoch, 22. August 2018 21:25 > >>> To: Thomas Darimont > >>> Cc: keycloak-dev > >>> Subject: Re: [keycloak-dev] Potential performance regression between > >>> 3.4.3.Final and 4.3.0.Final > >>> > >>> That's quite worrying. Is it limited to browsing users through the > admin > >>> console or are you seeing bad behaviour elsewhere? Same question > applies to > >>> the heap. My best bet here is that it has that something has changed > around > >>> user querying/caching. > >>> > >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < > >>> thomas.darimont at googlemail.com> wrote: > >>> > >>> > Hello Keycloak team, > >>> > > >>> > has anyone encountered some performance issues after upgrading 3.4.3 > >>> > to 4.x (4.3.0)? > >>> > > >>> > Today I noticed a performance regression while preparing an upgrade > >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. > >>> > > >>> > In our test environment, we have around ~100k test users stored in a > >>> > postgres-backed database. When we started the server with the new > >>> > Keycloak version, the migration went through, and everything looked > >>> > fine at first, but when we tried to browse the list of users via the > >>> > admin-console, we noticed that the CPU and memory consumption of the > >>> > server increased significantly, up to a point where Keycloak crashed > >>> with an OOME. > >>> > > >>> > All previous Keycloak versions including 3.4.3 were very modest with > >>> > their memory requirements and quite happy with ~1g heap. > >>> > However, that seems to have changed in Keycloak 4.3.0 - there we > >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. > >>> > Furthermore, we noticed that the response times for browsing the > >>> > paginated user view increased significantly as well: > >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In > >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation > >>> > takes > >>> > ~7 seconds for a test realm with just 10k users. > >>> > > >>> > In the test realm with 100k users, the time to load a single page in > >>> > the users listing was 66 seconds for version 4.3.0, on average - > >>> > compared to quite stable 80ms in 3.4.3. > >>> > > >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 > >>> > seconds for 100k users, so I assume the processing logic in Keycloak > >>> > is the culprit. > >>> > > >>> > The problem of long load-times can be reproduced with the Keycloak > >>> > docker images and the in-memory database. I also created a small > >>> > example project that creates some users with just a few attributes in > >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres > >>> > database to reproduce the problem. > >>> > https://github.com/thomasdarimont/kc-user-regression-tester > >>> > > >>> > Cheers, > >>> > Thomas > >>> > _______________________________________________ > >>> > keycloak-dev mailing list > >>> > keycloak-dev at lists.jboss.org > >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > >> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From Sebastian.Schuster at bosch-si.com Wed Aug 29 11:19:37 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Wed, 29 Aug 2018 15:19:37 +0000 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: Message-ID: <3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com> Thanks Hynek, I?ll test this. I also tracked the issue down to the ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel realm, UserModel user) method. However, what I am also seeing is when the method is running, it is emitting SQL queries that are not scoped down to the user id of the user being transformed. E.g. for the user attributes of a user the following query is generated: select attributes0_.USER_ID as USER_ID4_72_1_, attributes0_.ID as ID1_72_1_, attributes0_.ID as ID1_72_0_, attributes0_.NAME as NAME2_72_0_, attributes0_.USER_ID as USER_ID4_72_0_, attributes0_.VALUE as VALUE3_72_0_ from USER_ATTRIBUTE attributes0_ where attributes0_.USER_ID in ( select userentity0_.ID from USER_ENTITY userentity0_ where userentity0_.REALM_ID=? and ( userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null ) ) The subselect returns all real users from the realm. It is neither paged nor bound to a specific user id. The same happens in the above method for credentials and required actions. I am not a JPA/Hibernate expert so I might be missing something here? Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn From: Hynek Mlnarik Sent: Mittwoch, 29. August 2018 16:35 To: Thomas Darimont Cc: Stian Thorgersen ; keycloak-dev ; Schuster Sebastian (INST/ESY1) Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final Thanks Thomas and all, great help with reproducing the issue at least for admin user list endpoint. Could you please test [1] that it indeed fixes the issue? It was caused obtaining too much user details when unnecessary (for user list, only few fields are necessary). As for increased memory requirements, these might be unrelated to this fix, and we'll keep an eye on this. --Hynek [1] https://github.com/keycloak/keycloak/pull/5517 On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont > wrote: Hello, FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 Cheers, Thomas Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < sthorger at redhat.com>: > Can you open a bug for this in JIRA please and include as much details as > you can. > > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > >> Hi Sebastian, >> >> thanks for investigating! Unfortunately, I didn't have time yet to look >> further into the problem, but I also observed the slow attributes handling. >> Btw. we have around 100k users and ~700k attributes. The >> Entity-Attribute-Value data model takes it's toll here... >> >> @Stian so far I've only observed this when scrolling through the user >> overview in the admin-console, which also triggers >> the mentioned spike in heap usage. >> >> Cheers, >> Thomas >> >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1) >> >: >> >>> Btw. we also have 40K user attributes... >>> >>> Best regards, >>> Sebastian >>> >>> Mit freundlichen Gr??en / Best regards >>> >>> Dr.-Ing. Sebastian Schuster >>> >>> Engineering and Support (INST/ESY1) >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >>> GERMANY | www.bosch-si.com >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >>> Sebastian.Schuster at bosch-si.com >>> >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >>> Dr. Stefan Ferber, Michael Hahn >>> >>> >>> >>> >>> -----Original Message----- >>> From: keycloak-dev-bounces at lists.jboss.org < >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen >>> Sent: Mittwoch, 22. August 2018 21:25 >>> To: Thomas Darimont > >>> Cc: keycloak-dev > >>> Subject: Re: [keycloak-dev] Potential performance regression between >>> 3.4.3.Final and 4.3.0.Final >>> >>> That's quite worrying. Is it limited to browsing users through the admin >>> console or are you seeing bad behaviour elsewhere? Same question applies to >>> the heap. My best bet here is that it has that something has changed around >>> user querying/caching. >>> >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < >>> thomas.darimont at googlemail.com> wrote: >>> >>> > Hello Keycloak team, >>> > >>> > has anyone encountered some performance issues after upgrading 3.4.3 >>> > to 4.x (4.3.0)? >>> > >>> > Today I noticed a performance regression while preparing an upgrade >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. >>> > >>> > In our test environment, we have around ~100k test users stored in a >>> > postgres-backed database. When we started the server with the new >>> > Keycloak version, the migration went through, and everything looked >>> > fine at first, but when we tried to browse the list of users via the >>> > admin-console, we noticed that the CPU and memory consumption of the >>> > server increased significantly, up to a point where Keycloak crashed >>> with an OOME. >>> > >>> > All previous Keycloak versions including 3.4.3 were very modest with >>> > their memory requirements and quite happy with ~1g heap. >>> > However, that seems to have changed in Keycloak 4.3.0 - there we >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. >>> > Furthermore, we noticed that the response times for browsing the >>> > paginated user view increased significantly as well: >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation >>> > takes >>> > ~7 seconds for a test realm with just 10k users. >>> > >>> > In the test realm with 100k users, the time to load a single page in >>> > the users listing was 66 seconds for version 4.3.0, on average - >>> > compared to quite stable 80ms in 3.4.3. >>> > >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 >>> > seconds for 100k users, so I assume the processing logic in Keycloak >>> > is the culprit. >>> > >>> > The problem of long load-times can be reproduced with the Keycloak >>> > docker images and the in-memory database. I also created a small >>> > example project that creates some users with just a few attributes in >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres >>> > database to reproduce the problem. >>> > https://github.com/thomasdarimont/kc-user-regression-tester >>> > >>> > Cheers, >>> > Thomas >>> > _______________________________________________ >>> > keycloak-dev mailing list >>> > keycloak-dev at lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From thomas.darimont at googlemail.com Wed Aug 29 16:54:51 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Wed, 29 Aug 2018 22:54:51 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: <3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com> References: <3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com> Message-ID: Hello Hynek et al., I gave your PR 5517 a quick spin and tested the user listing in the admin-console on my Dell XPS (3.4GHz, 32g) laptop with local PostgreSQL backend. I can confirm that the response times for loading a single page in the user-listing are much better now with your fix: In a test realm with 10k users, loading a single page takes ~15ms compared to 8 seconds (!) before. The new query that is now generated whilst browsing the user list looks like this: Hibernate: select userentity0_.ID as ID1_75_, userentity0_.CREATED_TIMESTAMP as CREATED_2_75_, userentity0_.EMAIL as EMAIL3_75_, userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_75_, userentity0_.EMAIL_VERIFIED as EMAIL_VE5_75_, userentity0_.ENABLED as ENABLED6_75_, userentity0_.FEDERATION_LINK as FEDERATI7_75_, userentity0_.FIRST_NAME as FIRST_NA8_75_, userentity0_.LAST_NAME as LAST_NAM9_75_, userentity0_.NOT_BEFORE as NOT_BEF10_75_, userentity0_.REALM_ID as REALM_I11_75_, userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE12_75_, userentity0_.USERNAME as USERNAM13_75_ from USER_ENTITY userentity0_ where userentity0_.REALM_ID=? and ( userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null ) order by userentity0_.USERNAME limit ? offset ? The query looks reasonable with appropriate limits applied. However, it seems that policies are fetched for each returned user via a dedicated query, e.g.: select resourcese0_.ID as ID1_64_0_, resourcese0_.ALLOW_RS_REMOTE_MGMT as ALLOW_RS2_64_0_, resourcese0_.POLICY_ENFORCE_MODE as POLICY_E3_64_0_ from RESOURCE_SERVER resourcese0_ where resourcese0_.ID=? Would it be possible to load those in one batch? Cheers, Thomas Am Mi., 29. Aug. 2018 um 17:19 Uhr schrieb Schuster Sebastian (INST/ESY1) < Sebastian.Schuster at bosch-si.com>: > Thanks Hynek, I?ll test this. I also tracked the issue down to the > ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel > realm, UserModel user) method. However, what I am also seeing is when the > method is running, it is emitting SQL queries that are not scoped down to > the user id of the user being transformed. E.g. for the user attributes of > a user the following query is generated: > > > > select > > attributes0_.USER_ID as USER_ID4_72_1_, > > attributes0_.ID as ID1_72_1_, > > attributes0_.ID as ID1_72_0_, > > attributes0_.NAME as NAME2_72_0_, > > attributes0_.USER_ID as USER_ID4_72_0_, > > attributes0_.VALUE as VALUE3_72_0_ > > from > > USER_ATTRIBUTE attributes0_ > > where > > attributes0_.USER_ID in ( > > select > > userentity0_.ID > > from > > USER_ENTITY userentity0_ > > where > > userentity0_.REALM_ID=? > > and ( > > userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null > > ) > > ) > > > > The subselect returns all real users from the realm. It is neither paged > nor bound to a specific user id. The same happens in the above method for > credentials and required actions. I am not a JPA/Hibernate expert so I > might be missing something here? > > > > Best regards, > > Sebastian > > > > Mit freundlichen Gr??en / Best regards > > > *Dr.-Ing. Sebastian Schuster * > Engineering and Support (INST/ESY1) > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > GERMANY | www.bosch-si.com > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > Sebastian.Schuster at bosch-si.com > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > Stefan Ferber, Michael Hahn > > > > *From:* Hynek Mlnarik > *Sent:* Mittwoch, 29. August 2018 16:35 > *To:* Thomas Darimont > *Cc:* Stian Thorgersen ; keycloak-dev < > keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> > *Subject:* Re: [keycloak-dev] Potential performance regression between > 3.4.3.Final and 4.3.0.Final > > > > Thanks Thomas and all, > > > > great help with reproducing the issue at least for admin user list > endpoint. Could you please test [1] that it indeed fixes the issue? It was > caused obtaining too much user details when unnecessary (for user list, > only few fields are necessary). > > > > As for increased memory requirements, these might be unrelated to this > fix, and we'll keep an eye on this. > > > > --Hynek > > > > [1] https://github.com/keycloak/keycloak/pull/5517 > > > > > > > > On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > > Hello, > > FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 > > Cheers, > Thomas > > Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < > sthorger at redhat.com>: > > > Can you open a bug for this in JIRA please and include as much details as > > you can. > > > > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < > > thomas.darimont at googlemail.com> wrote: > > > >> Hi Sebastian, > >> > >> thanks for investigating! Unfortunately, I didn't have time yet to look > >> further into the problem, but I also observed the slow attributes > handling. > >> Btw. we have around 100k users and ~700k attributes. The > >> Entity-Attribute-Value data model takes it's toll here... > >> > >> @Stian so far I've only observed this when scrolling through the user > >> overview in the admin-console, which also triggers > >> the mentioned spike in heap usage. > >> > >> Cheers, > >> Thomas > >> > >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian > (INST/ESY1) > >> : > >> > >>> Btw. we also have 40K user attributes... > >>> > >>> Best regards, > >>> Sebastian > >>> > >>> Mit freundlichen Gr??en / Best regards > >>> > >>> Dr.-Ing. Sebastian Schuster > >>> > >>> Engineering and Support (INST/ESY1) > >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > >>> GERMANY | www.bosch-si.com > >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > >>> Sebastian.Schuster at bosch-si.com > >>> > >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: > >>> Dr. Stefan Ferber, Michael Hahn > >>> > >>> > >>> > >>> > >>> -----Original Message----- > >>> From: keycloak-dev-bounces at lists.jboss.org < > >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen > >>> Sent: Mittwoch, 22. August 2018 21:25 > >>> To: Thomas Darimont > >>> Cc: keycloak-dev > >>> Subject: Re: [keycloak-dev] Potential performance regression between > >>> 3.4.3.Final and 4.3.0.Final > >>> > >>> That's quite worrying. Is it limited to browsing users through the > admin > >>> console or are you seeing bad behaviour elsewhere? Same question > applies to > >>> the heap. My best bet here is that it has that something has changed > around > >>> user querying/caching. > >>> > >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < > >>> thomas.darimont at googlemail.com> wrote: > >>> > >>> > Hello Keycloak team, > >>> > > >>> > has anyone encountered some performance issues after upgrading 3.4.3 > >>> > to 4.x (4.3.0)? > >>> > > >>> > Today I noticed a performance regression while preparing an upgrade > >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. > >>> > > >>> > In our test environment, we have around ~100k test users stored in a > >>> > postgres-backed database. When we started the server with the new > >>> > Keycloak version, the migration went through, and everything looked > >>> > fine at first, but when we tried to browse the list of users via the > >>> > admin-console, we noticed that the CPU and memory consumption of the > >>> > server increased significantly, up to a point where Keycloak crashed > >>> with an OOME. > >>> > > >>> > All previous Keycloak versions including 3.4.3 were very modest with > >>> > their memory requirements and quite happy with ~1g heap. > >>> > However, that seems to have changed in Keycloak 4.3.0 - there we > >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. > >>> > Furthermore, we noticed that the response times for browsing the > >>> > paginated user view increased significantly as well: > >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In > >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation > >>> > takes > >>> > ~7 seconds for a test realm with just 10k users. > >>> > > >>> > In the test realm with 100k users, the time to load a single page in > >>> > the users listing was 66 seconds for version 4.3.0, on average - > >>> > compared to quite stable 80ms in 3.4.3. > >>> > > >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 > >>> > seconds for 100k users, so I assume the processing logic in Keycloak > >>> > is the culprit. > >>> > > >>> > The problem of long load-times can be reproduced with the Keycloak > >>> > docker images and the in-memory database. I also created a small > >>> > example project that creates some users with just a few attributes in > >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres > >>> > database to reproduce the problem. > >>> > https://github.com/thomasdarimont/kc-user-regression-tester > >>> > > >>> > Cheers, > >>> > Thomas > >>> > _______________________________________________ > >>> > keycloak-dev mailing list > >>> > keycloak-dev at lists.jboss.org > >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > >> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > From thomas.darimont at googlemail.com Wed Aug 29 17:29:10 2018 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Wed, 29 Aug 2018 23:29:10 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: <3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com> Message-ID: ... forgot to mention: if you want to try this as well, remember to clear your browser cache for the admin-console... Otherwise, you might still see slow load-times, since you run the old js code which doesn't yet use the new "briefRepresentation" attribute in the UserListCtrl ( $scope.query.briefRepresentation = 'true'; ), which controls the use of a faster overview query. Am Mi., 29. Aug. 2018 um 22:54 Uhr schrieb Thomas Darimont < thomas.darimont at googlemail.com>: > Hello Hynek et al., > > I gave your PR 5517 a quick spin and tested the user listing in the > admin-console on my Dell XPS (3.4GHz, 32g) laptop with local PostgreSQL > backend. > I can confirm that the response times for loading a single page in the > user-listing are much better now with your fix: > In a test realm with 10k users, loading a single page takes ~15ms compared > to 8 seconds (!) before. > > The new query that is now generated whilst browsing the user list looks > like this: > > Hibernate: > select > userentity0_.ID as ID1_75_, > userentity0_.CREATED_TIMESTAMP as CREATED_2_75_, > userentity0_.EMAIL as EMAIL3_75_, > userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_75_, > userentity0_.EMAIL_VERIFIED as EMAIL_VE5_75_, > userentity0_.ENABLED as ENABLED6_75_, > userentity0_.FEDERATION_LINK as FEDERATI7_75_, > userentity0_.FIRST_NAME as FIRST_NA8_75_, > userentity0_.LAST_NAME as LAST_NAM9_75_, > userentity0_.NOT_BEFORE as NOT_BEF10_75_, > userentity0_.REALM_ID as REALM_I11_75_, > userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE12_75_, > userentity0_.USERNAME as USERNAM13_75_ > from > USER_ENTITY userentity0_ > where > userentity0_.REALM_ID=? > and ( > userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null > ) > order by > userentity0_.USERNAME limit ? offset ? > > The query looks reasonable with appropriate limits applied. > > However, it seems that policies are fetched for each returned user via a > dedicated query, e.g.: > > select > resourcese0_.ID as ID1_64_0_, > resourcese0_.ALLOW_RS_REMOTE_MGMT as ALLOW_RS2_64_0_, > resourcese0_.POLICY_ENFORCE_MODE as POLICY_E3_64_0_ > from > RESOURCE_SERVER resourcese0_ > where > resourcese0_.ID=? > > Would it be possible to load those in one batch? > > Cheers, > Thomas > > Am Mi., 29. Aug. 2018 um 17:19 Uhr schrieb Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com>: > >> Thanks Hynek, I?ll test this. I also tracked the issue down to the >> ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel >> realm, UserModel user) method. However, what I am also seeing is when the >> method is running, it is emitting SQL queries that are not scoped down to >> the user id of the user being transformed. E.g. for the user attributes of >> a user the following query is generated: >> >> >> >> select >> >> attributes0_.USER_ID as USER_ID4_72_1_, >> >> attributes0_.ID as ID1_72_1_, >> >> attributes0_.ID as ID1_72_0_, >> >> attributes0_.NAME as NAME2_72_0_, >> >> attributes0_.USER_ID as USER_ID4_72_0_, >> >> attributes0_.VALUE as VALUE3_72_0_ >> >> from >> >> USER_ATTRIBUTE attributes0_ >> >> where >> >> attributes0_.USER_ID in ( >> >> select >> >> userentity0_.ID >> >> from >> >> USER_ENTITY userentity0_ >> >> where >> >> userentity0_.REALM_ID=? >> >> and ( >> >> userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null >> >> ) >> >> ) >> >> >> >> The subselect returns all real users from the realm. It is neither paged >> nor bound to a specific user id. The same happens in the above method for >> credentials and required actions. I am not a JPA/Hibernate expert so I >> might be missing something here? >> >> >> >> Best regards, >> >> Sebastian >> >> >> >> Mit freundlichen Gr??en / Best regards >> >> >> *Dr.-Ing. Sebastian Schuster * >> Engineering and Support (INST/ESY1) >> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> GERMANY | www.bosch-si.com >> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> Sebastian.Schuster at bosch-si.com >> >> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >> Dr. Stefan Ferber, Michael Hahn >> >> >> >> *From:* Hynek Mlnarik >> *Sent:* Mittwoch, 29. August 2018 16:35 >> *To:* Thomas Darimont >> *Cc:* Stian Thorgersen ; keycloak-dev < >> keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) < >> Sebastian.Schuster at bosch-si.com> >> *Subject:* Re: [keycloak-dev] Potential performance regression between >> 3.4.3.Final and 4.3.0.Final >> >> >> >> Thanks Thomas and all, >> >> >> >> great help with reproducing the issue at least for admin user list >> endpoint. Could you please test [1] that it indeed fixes the issue? It was >> caused obtaining too much user details when unnecessary (for user list, >> only few fields are necessary). >> >> >> >> As for increased memory requirements, these might be unrelated to this >> fix, and we'll keep an eye on this. >> >> >> >> --Hynek >> >> >> >> [1] https://github.com/keycloak/keycloak/pull/5517 >> >> >> >> >> >> >> >> On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont < >> thomas.darimont at googlemail.com> wrote: >> >> Hello, >> >> FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 >> >> Cheers, >> Thomas >> >> Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < >> sthorger at redhat.com>: >> >> > Can you open a bug for this in JIRA please and include as much details >> as >> > you can. >> > >> > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < >> > thomas.darimont at googlemail.com> wrote: >> > >> >> Hi Sebastian, >> >> >> >> thanks for investigating! Unfortunately, I didn't have time yet to look >> >> further into the problem, but I also observed the slow attributes >> handling. >> >> Btw. we have around 100k users and ~700k attributes. The >> >> Entity-Attribute-Value data model takes it's toll here... >> >> >> >> @Stian so far I've only observed this when scrolling through the user >> >> overview in the admin-console, which also triggers >> >> the mentioned spike in heap usage. >> >> >> >> Cheers, >> >> Thomas >> >> >> >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian >> (INST/ESY1) >> >> : >> >> >> >>> Btw. we also have 40K user attributes... >> >>> >> >>> Best regards, >> >>> Sebastian >> >>> >> >>> Mit freundlichen Gr??en / Best regards >> >>> >> >>> Dr.-Ing. Sebastian Schuster >> >>> >> >>> Engineering and Support (INST/ESY1) >> >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> >>> GERMANY | www.bosch-si.com >> >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> >>> Sebastian.Schuster at bosch-si.com >> >>> >> >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 >> B >> >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >> >>> Dr. Stefan Ferber, Michael Hahn >> >>> >> >>> >> >>> >> >>> >> >>> -----Original Message----- >> >>> From: keycloak-dev-bounces at lists.jboss.org < >> >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen >> >>> Sent: Mittwoch, 22. August 2018 21:25 >> >>> To: Thomas Darimont >> >>> Cc: keycloak-dev >> >>> Subject: Re: [keycloak-dev] Potential performance regression between >> >>> 3.4.3.Final and 4.3.0.Final >> >>> >> >>> That's quite worrying. Is it limited to browsing users through the >> admin >> >>> console or are you seeing bad behaviour elsewhere? Same question >> applies to >> >>> the heap. My best bet here is that it has that something has changed >> around >> >>> user querying/caching. >> >>> >> >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < >> >>> thomas.darimont at googlemail.com> wrote: >> >>> >> >>> > Hello Keycloak team, >> >>> > >> >>> > has anyone encountered some performance issues after upgrading 3.4.3 >> >>> > to 4.x (4.3.0)? >> >>> > >> >>> > Today I noticed a performance regression while preparing an upgrade >> >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. >> >>> > >> >>> > In our test environment, we have around ~100k test users stored in a >> >>> > postgres-backed database. When we started the server with the new >> >>> > Keycloak version, the migration went through, and everything looked >> >>> > fine at first, but when we tried to browse the list of users via the >> >>> > admin-console, we noticed that the CPU and memory consumption of the >> >>> > server increased significantly, up to a point where Keycloak crashed >> >>> with an OOME. >> >>> > >> >>> > All previous Keycloak versions including 3.4.3 were very modest with >> >>> > their memory requirements and quite happy with ~1g heap. >> >>> > However, that seems to have changed in Keycloak 4.3.0 - there we >> >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. >> >>> > Furthermore, we noticed that the response times for browsing the >> >>> > paginated user view increased significantly as well: >> >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In >> >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same >> operation >> >>> > takes >> >>> > ~7 seconds for a test realm with just 10k users. >> >>> > >> >>> > In the test realm with 100k users, the time to load a single page in >> >>> > the users listing was 66 seconds for version 4.3.0, on average - >> >>> > compared to quite stable 80ms in 3.4.3. >> >>> > >> >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 >> >>> > seconds for 100k users, so I assume the processing logic in Keycloak >> >>> > is the culprit. >> >>> > >> >>> > The problem of long load-times can be reproduced with the Keycloak >> >>> > docker images and the in-memory database. I also created a small >> >>> > example project that creates some users with just a few attributes >> in >> >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres >> >>> > database to reproduce the problem. >> >>> > https://github.com/thomasdarimont/kc-user-regression-tester >> >>> > >> >>> > Cheers, >> >>> > Thomas >> >>> > _______________________________________________ >> >>> > keycloak-dev mailing list >> >>> > keycloak-dev at lists.jboss.org >> >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >>> > >> >>> _______________________________________________ >> >>> keycloak-dev mailing list >> >>> keycloak-dev at lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >>> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> From mposolda at redhat.com Thu Aug 30 05:42:26 2018 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 30 Aug 2018 11:42:26 +0200 Subject: [keycloak-dev] Infinispan Cache and Keycloak instance restart In-Reply-To: References: Message-ID: <707073d4-f582-629b-d5a5-017d91aac3c2@redhat.com> Hi, how many owners are on your infinispan distributed caches? If it's 1 owner (default setting), the failover is not available and the behaviour you're seeing is hence expected. You may need to change owners to 2 or 3. See documentation for more details (Also latest docs coming with Keycloak 4.4.X release contains some more details around this). Marek On 28/08/18 11:39, Adrien DESBIAUX wrote: > Hello! > > > Thanks again for maintaining Keycloak. > > > I would have a question: > > > - We are running Keycloak 3.4 > > - We did setup a Keycloak standalone HA cluster (3 nodes). > > - As well we did connect those nodes to an Infinispan cluster. > > > All is running fine. However the trouble comes when we want to update the Keycloak Nodes. > > Updates can be either on themes or modules. > > > We update for example a module, e.g a new user federation SPI, with Ansible. > > Then we restart the Keycloak nodes one after another. The restart is required so that a Keycloak node can register the new module. > > > All is correct except that users need to login again after the reload has been performed. > > Is it something we can prevent happening? it is a strange behaviour, since we rely on Infinispan for sessions cache... > > > Is there another local cache to be configured in Infinispan to avoid users to re-login after restarting Keycloak? > > > It would be very helpful if you could give us a direction to investigate as rolling out several updates per day would lead to several re-logins from a user perspective. > > > Cheers, > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From Sebastian.Schuster at bosch-si.com Thu Aug 30 06:06:37 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Thu, 30 Aug 2018 10:06:37 +0000 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: <3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com> Message-ID: Hi Thomas, Your query is the same as before, it is the initial retrieval of the user entity. The query I am referring to is the one loading the user attributes after the user entity has been fetched as part of ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel realm, UserModel user). It just looks wrong to me that all users are fetched when only the attributes of a single user are needed. However, with Hynek?s change, the user attributes are no longer fetched because only a BriefRepresentation is requested, that?s why it is faster now. I still think a full representation should also be faster and not fetch all users from a realm. Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn From: Thomas Darimont Sent: Mittwoch, 29. August 2018 22:55 To: Schuster Sebastian (INST/ESY1) Cc: Hynek Mlnarik ; Stian Thorgersen ; keycloak-dev Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final Hello Hynek et al., I gave your PR 5517 a quick spin and tested the user listing in the admin-console on my Dell XPS (3.4GHz, 32g) laptop with local PostgreSQL backend. I can confirm that the response times for loading a single page in the user-listing are much better now with your fix: In a test realm with 10k users, loading a single page takes ~15ms compared to 8 seconds (!) before. The new query that is now generated whilst browsing the user list looks like this: Hibernate: select userentity0_.ID as ID1_75_, userentity0_.CREATED_TIMESTAMP as CREATED_2_75_, userentity0_.EMAIL as EMAIL3_75_, userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_75_, userentity0_.EMAIL_VERIFIED as EMAIL_VE5_75_, userentity0_.ENABLED as ENABLED6_75_, userentity0_.FEDERATION_LINK as FEDERATI7_75_, userentity0_.FIRST_NAME as FIRST_NA8_75_, userentity0_.LAST_NAME as LAST_NAM9_75_, userentity0_.NOT_BEFORE as NOT_BEF10_75_, userentity0_.REALM_ID as REALM_I11_75_, userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE12_75_, userentity0_.USERNAME as USERNAM13_75_ from USER_ENTITY userentity0_ where userentity0_.REALM_ID=? and ( userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null ) order by userentity0_.USERNAME limit ? offset ? The query looks reasonable with appropriate limits applied. However, it seems that policies are fetched for each returned user via a dedicated query, e.g.: select resourcese0_.ID as ID1_64_0_, resourcese0_.ALLOW_RS_REMOTE_MGMT as ALLOW_RS2_64_0_, resourcese0_.POLICY_ENFORCE_MODE as POLICY_E3_64_0_ from RESOURCE_SERVER resourcese0_ where resourcese0_.ID=? Would it be possible to load those in one batch? Cheers, Thomas Am Mi., 29. Aug. 2018 um 17:19 Uhr schrieb Schuster Sebastian (INST/ESY1) >: Thanks Hynek, I?ll test this. I also tracked the issue down to the ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel realm, UserModel user) method. However, what I am also seeing is when the method is running, it is emitting SQL queries that are not scoped down to the user id of the user being transformed. E.g. for the user attributes of a user the following query is generated: select attributes0_.USER_ID as USER_ID4_72_1_, attributes0_.ID as ID1_72_1_, attributes0_.ID as ID1_72_0_, attributes0_.NAME as NAME2_72_0_, attributes0_.USER_ID as USER_ID4_72_0_, attributes0_.VALUE as VALUE3_72_0_ from USER_ATTRIBUTE attributes0_ where attributes0_.USER_ID in ( select userentity0_.ID from USER_ENTITY userentity0_ where userentity0_.REALM_ID=? and ( userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null ) ) The subselect returns all real users from the realm. It is neither paged nor bound to a specific user id. The same happens in the above method for credentials and required actions. I am not a JPA/Hibernate expert so I might be missing something here? Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn From: Hynek Mlnarik > Sent: Mittwoch, 29. August 2018 16:35 To: Thomas Darimont > Cc: Stian Thorgersen >; keycloak-dev >; Schuster Sebastian (INST/ESY1) > Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final Thanks Thomas and all, great help with reproducing the issue at least for admin user list endpoint. Could you please test [1] that it indeed fixes the issue? It was caused obtaining too much user details when unnecessary (for user list, only few fields are necessary). As for increased memory requirements, these might be unrelated to this fix, and we'll keep an eye on this. --Hynek [1] https://github.com/keycloak/keycloak/pull/5517 On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont > wrote: Hello, FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 Cheers, Thomas Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < sthorger at redhat.com>: > Can you open a bug for this in JIRA please and include as much details as > you can. > > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > >> Hi Sebastian, >> >> thanks for investigating! Unfortunately, I didn't have time yet to look >> further into the problem, but I also observed the slow attributes handling. >> Btw. we have around 100k users and ~700k attributes. The >> Entity-Attribute-Value data model takes it's toll here... >> >> @Stian so far I've only observed this when scrolling through the user >> overview in the admin-console, which also triggers >> the mentioned spike in heap usage. >> >> Cheers, >> Thomas >> >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1) >> >: >> >>> Btw. we also have 40K user attributes... >>> >>> Best regards, >>> Sebastian >>> >>> Mit freundlichen Gr??en / Best regards >>> >>> Dr.-Ing. Sebastian Schuster >>> >>> Engineering and Support (INST/ESY1) >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >>> GERMANY | www.bosch-si.com >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >>> Sebastian.Schuster at bosch-si.com >>> >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >>> Dr. Stefan Ferber, Michael Hahn >>> >>> >>> >>> >>> -----Original Message----- >>> From: keycloak-dev-bounces at lists.jboss.org < >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen >>> Sent: Mittwoch, 22. August 2018 21:25 >>> To: Thomas Darimont > >>> Cc: keycloak-dev > >>> Subject: Re: [keycloak-dev] Potential performance regression between >>> 3.4.3.Final and 4.3.0.Final >>> >>> That's quite worrying. Is it limited to browsing users through the admin >>> console or are you seeing bad behaviour elsewhere? Same question applies to >>> the heap. My best bet here is that it has that something has changed around >>> user querying/caching. >>> >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < >>> thomas.darimont at googlemail.com> wrote: >>> >>> > Hello Keycloak team, >>> > >>> > has anyone encountered some performance issues after upgrading 3.4.3 >>> > to 4.x (4.3.0)? >>> > >>> > Today I noticed a performance regression while preparing an upgrade >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. >>> > >>> > In our test environment, we have around ~100k test users stored in a >>> > postgres-backed database. When we started the server with the new >>> > Keycloak version, the migration went through, and everything looked >>> > fine at first, but when we tried to browse the list of users via the >>> > admin-console, we noticed that the CPU and memory consumption of the >>> > server increased significantly, up to a point where Keycloak crashed >>> with an OOME. >>> > >>> > All previous Keycloak versions including 3.4.3 were very modest with >>> > their memory requirements and quite happy with ~1g heap. >>> > However, that seems to have changed in Keycloak 4.3.0 - there we >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. >>> > Furthermore, we noticed that the response times for browsing the >>> > paginated user view increased significantly as well: >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation >>> > takes >>> > ~7 seconds for a test realm with just 10k users. >>> > >>> > In the test realm with 100k users, the time to load a single page in >>> > the users listing was 66 seconds for version 4.3.0, on average - >>> > compared to quite stable 80ms in 3.4.3. >>> > >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 >>> > seconds for 100k users, so I assume the processing logic in Keycloak >>> > is the culprit. >>> > >>> > The problem of long load-times can be reproduced with the Keycloak >>> > docker images and the in-memory database. I also created a small >>> > example project that creates some users with just a few attributes in >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres >>> > database to reproduce the problem. >>> > https://github.com/thomasdarimont/kc-user-regression-tester >>> > >>> > Cheers, >>> > Thomas >>> > _______________________________________________ >>> > keycloak-dev mailing list >>> > keycloak-dev at lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From hmlnarik at redhat.com Thu Aug 30 06:58:25 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Thu, 30 Aug 2018 12:58:25 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: <3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com> Message-ID: Hi Sebastian, Thomas, I agree that this query is bad performance-wise. It is caused inability of database to easily limit rows in the resultset of query that joins users and attributes (and other tables) if there might are several attributes per user while we want to return exactly n users (the LIMIT/OFFSET won't work here for parent table due to joining with child records). Hibernate thus resorts to its own filtering of the full resultset. That is one option, the other is having subsequent selects and limiting batch size. Both approaches have pros and cons, and actually the subsequent selects were changed to subselects precisely because of performance. Looks like we might need to revert back to the select approach and tweak batch size. That will require some time for optimizations. For now, the most painful issue with user list seems resolved and thus I'm merging the PR. There will be subsequent work though on further optimizations to user / user list retrieval, that relates to both your suggestion and Thomas's comment (that relates to authz resource server store caching). Thank you both for your help! --Hynek On Thu, Aug 30, 2018 at 12:06 PM Schuster Sebastian (INST/ESY1) < Sebastian.Schuster at bosch-si.com> wrote: > Hi Thomas, > > > > Your query is the same as before, it is the initial retrieval of the user > entity. The query I am referring to is the one loading the user attributes > after the user entity has been fetched as part of > ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel > realm, UserModel user). It just looks wrong to me that all users are > fetched when only the attributes of a single user are needed. However, with > Hynek?s change, the user attributes are no longer fetched because only a > BriefRepresentation is requested, that?s why it is faster now. I still > think a full representation should also be faster and not fetch all users > from a realm. > > > > Best regards, > > Sebastian > > > > Mit freundlichen Gr??en / Best regards > > > *Dr.-Ing. Sebastian Schuster * > Engineering and Support (INST/ESY1) > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > GERMANY | www.bosch-si.com > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > Sebastian.Schuster at bosch-si.com > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > Stefan Ferber, Michael Hahn > > > > *From:* Thomas Darimont > *Sent:* Mittwoch, 29. August 2018 22:55 > *To:* Schuster Sebastian (INST/ESY1) > *Cc:* Hynek Mlnarik ; Stian Thorgersen < > stian at redhat.com>; keycloak-dev > *Subject:* Re: [keycloak-dev] Potential performance regression between > 3.4.3.Final and 4.3.0.Final > > > > Hello Hynek et al., > > > > I gave your PR 5517 a quick spin and tested the user listing in the > admin-console on my Dell XPS (3.4GHz, 32g) laptop with local PostgreSQL > backend. > > I can confirm that the response times for loading a single page in the > user-listing are much better now with your fix: > > In a test realm with 10k users, loading a single page takes ~15ms compared > to 8 seconds (!) before. > > > > The new query that is now generated whilst browsing the user list looks > like this: > > > > Hibernate: > > select > > userentity0_.ID as ID1_75_, > > userentity0_.CREATED_TIMESTAMP as CREATED_2_75_, > > userentity0_.EMAIL as EMAIL3_75_, > > userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_75_, > > userentity0_.EMAIL_VERIFIED as EMAIL_VE5_75_, > > userentity0_.ENABLED as ENABLED6_75_, > > userentity0_.FEDERATION_LINK as FEDERATI7_75_, > > userentity0_.FIRST_NAME as FIRST_NA8_75_, > > userentity0_.LAST_NAME as LAST_NAM9_75_, > > userentity0_.NOT_BEFORE as NOT_BEF10_75_, > > userentity0_.REALM_ID as REALM_I11_75_, > > userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE12_75_, > > userentity0_.USERNAME as USERNAM13_75_ > > from > > USER_ENTITY userentity0_ > > where > > userentity0_.REALM_ID=? > > and ( > > userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null > > ) > > order by > > userentity0_.USERNAME limit ? offset ? > > > > The query looks reasonable with appropriate limits applied. > > > > However, it seems that policies are fetched for each returned user via a > dedicated query, e.g.: > > > > select > > resourcese0_.ID as ID1_64_0_, > > resourcese0_.ALLOW_RS_REMOTE_MGMT as ALLOW_RS2_64_0_, > > resourcese0_.POLICY_ENFORCE_MODE as POLICY_E3_64_0_ > > from > > RESOURCE_SERVER resourcese0_ > > where > > resourcese0_.ID=? > > > > Would it be possible to load those in one batch? > > > > Cheers, > > Thomas > > > > Am Mi., 29. Aug. 2018 um 17:19 Uhr schrieb Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com>: > > Thanks Hynek, I?ll test this. I also tracked the issue down to the > ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel > realm, UserModel user) method. However, what I am also seeing is when the > method is running, it is emitting SQL queries that are not scoped down to > the user id of the user being transformed. E.g. for the user attributes of > a user the following query is generated: > > > > select > > attributes0_.USER_ID as USER_ID4_72_1_, > > attributes0_.ID as ID1_72_1_, > > attributes0_.ID as ID1_72_0_, > > attributes0_.NAME as NAME2_72_0_, > > attributes0_.USER_ID as USER_ID4_72_0_, > > attributes0_.VALUE as VALUE3_72_0_ > > from > > USER_ATTRIBUTE attributes0_ > > where > > attributes0_.USER_ID in ( > > select > > userentity0_.ID > > from > > USER_ENTITY userentity0_ > > where > > userentity0_.REALM_ID=? > > and ( > > userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null > > ) > > ) > > > > The subselect returns all real users from the realm. It is neither paged > nor bound to a specific user id. The same happens in the above method for > credentials and required actions. I am not a JPA/Hibernate expert so I > might be missing something here? > > > > Best regards, > > Sebastian > > > > Mit freundlichen Gr??en / Best regards > > > *Dr.-Ing. Sebastian Schuster * > Engineering and Support (INST/ESY1) > Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > GERMANY | www.bosch-si.com > Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > Sebastian.Schuster at bosch-si.com > > Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. > Stefan Ferber, Michael Hahn > > > *From:* Hynek Mlnarik > *Sent:* Mittwoch, 29. August 2018 16:35 > *To:* Thomas Darimont > *Cc:* Stian Thorgersen ; keycloak-dev < > keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> > *Subject:* Re: [keycloak-dev] Potential performance regression between > 3.4.3.Final and 4.3.0.Final > > > > Thanks Thomas and all, > > > > great help with reproducing the issue at least for admin user list > endpoint. Could you please test [1] that it indeed fixes the issue? It was > caused obtaining too much user details when unnecessary (for user list, > only few fields are necessary). > > > > As for increased memory requirements, these might be unrelated to this > fix, and we'll keep an eye on this. > > > > --Hynek > > > > [1] https://github.com/keycloak/keycloak/pull/5517 > > > > > > > > On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > > Hello, > > FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 > > Cheers, > Thomas > > Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < > sthorger at redhat.com>: > > > Can you open a bug for this in JIRA please and include as much details as > > you can. > > > > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < > > thomas.darimont at googlemail.com> wrote: > > > >> Hi Sebastian, > >> > >> thanks for investigating! Unfortunately, I didn't have time yet to look > >> further into the problem, but I also observed the slow attributes > handling. > >> Btw. we have around 100k users and ~700k attributes. The > >> Entity-Attribute-Value data model takes it's toll here... > >> > >> @Stian so far I've only observed this when scrolling through the user > >> overview in the admin-console, which also triggers > >> the mentioned spike in heap usage. > >> > >> Cheers, > >> Thomas > >> > >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian > (INST/ESY1) > >> : > >> > >>> Btw. we also have 40K user attributes... > >>> > >>> Best regards, > >>> Sebastian > >>> > >>> Mit freundlichen Gr??en / Best regards > >>> > >>> Dr.-Ing. Sebastian Schuster > >>> > >>> Engineering and Support (INST/ESY1) > >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | > >>> GERMANY | www.bosch-si.com > >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | > >>> Sebastian.Schuster at bosch-si.com > >>> > >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B > >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: > >>> Dr. Stefan Ferber, Michael Hahn > >>> > >>> > >>> > >>> > >>> -----Original Message----- > >>> From: keycloak-dev-bounces at lists.jboss.org < > >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen > >>> Sent: Mittwoch, 22. August 2018 21:25 > >>> To: Thomas Darimont > >>> Cc: keycloak-dev > >>> Subject: Re: [keycloak-dev] Potential performance regression between > >>> 3.4.3.Final and 4.3.0.Final > >>> > >>> That's quite worrying. Is it limited to browsing users through the > admin > >>> console or are you seeing bad behaviour elsewhere? Same question > applies to > >>> the heap. My best bet here is that it has that something has changed > around > >>> user querying/caching. > >>> > >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < > >>> thomas.darimont at googlemail.com> wrote: > >>> > >>> > Hello Keycloak team, > >>> > > >>> > has anyone encountered some performance issues after upgrading 3.4.3 > >>> > to 4.x (4.3.0)? > >>> > > >>> > Today I noticed a performance regression while preparing an upgrade > >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. > >>> > > >>> > In our test environment, we have around ~100k test users stored in a > >>> > postgres-backed database. When we started the server with the new > >>> > Keycloak version, the migration went through, and everything looked > >>> > fine at first, but when we tried to browse the list of users via the > >>> > admin-console, we noticed that the CPU and memory consumption of the > >>> > server increased significantly, up to a point where Keycloak crashed > >>> with an OOME. > >>> > > >>> > All previous Keycloak versions including 3.4.3 were very modest with > >>> > their memory requirements and quite happy with ~1g heap. > >>> > However, that seems to have changed in Keycloak 4.3.0 - there we > >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. > >>> > Furthermore, we noticed that the response times for browsing the > >>> > paginated user view increased significantly as well: > >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In > >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation > >>> > takes > >>> > ~7 seconds for a test realm with just 10k users. > >>> > > >>> > In the test realm with 100k users, the time to load a single page in > >>> > the users listing was 66 seconds for version 4.3.0, on average - > >>> > compared to quite stable 80ms in 3.4.3. > >>> > > >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 > >>> > seconds for 100k users, so I assume the processing logic in Keycloak > >>> > is the culprit. > >>> > > >>> > The problem of long load-times can be reproduced with the Keycloak > >>> > docker images and the in-memory database. I also created a small > >>> > example project that creates some users with just a few attributes in > >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres > >>> > database to reproduce the problem. > >>> > https://github.com/thomasdarimont/kc-user-regression-tester > >>> > > >>> > Cheers, > >>> > Thomas > >>> > _______________________________________________ > >>> > keycloak-dev mailing list > >>> > keycloak-dev at lists.jboss.org > >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > >> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > From adesbiaux at vente-privee.com Thu Aug 30 10:56:23 2018 From: adesbiaux at vente-privee.com (Adrien DESBIAUX) Date: Thu, 30 Aug 2018 14:56:23 +0000 Subject: [keycloak-dev] Infinispan Cache and Keycloak instance restart In-Reply-To: <707073d4-f582-629b-d5a5-017d91aac3c2@redhat.com> References: , <707073d4-f582-629b-d5a5-017d91aac3c2@redhat.com> Message-ID: Hi, Many thanks Marek. We will def have a look. Cheers, ________________________________ From: Marek Posolda Sent: Thursday, August 30, 2018 11:42:26 AM To: Adrien DESBIAUX; keycloak-dev at lists.jboss.org Subject: Re: [keycloak-dev] Infinispan Cache and Keycloak instance restart Hi, how many owners are on your infinispan distributed caches? If it's 1 owner (default setting), the failover is not available and the behaviour you're seeing is hence expected. You may need to change owners to 2 or 3. See documentation for more details (Also latest docs coming with Keycloak 4.4.X release contains some more details around this). Marek On 28/08/18 11:39, Adrien DESBIAUX wrote: > Hello! > > > Thanks again for maintaining Keycloak. > > > I would have a question: > > > - We are running Keycloak 3.4 > > - We did setup a Keycloak standalone HA cluster (3 nodes). > > - As well we did connect those nodes to an Infinispan cluster. > > > All is running fine. However the trouble comes when we want to update the Keycloak Nodes. > > Updates can be either on themes or modules. > > > We update for example a module, e.g a new user federation SPI, with Ansible. > > Then we restart the Keycloak nodes one after another. The restart is required so that a Keycloak node can register the new module. > > > All is correct except that users need to login again after the reload has been performed. > > Is it something we can prevent happening? it is a strange behaviour, since we rely on Infinispan for sessions cache... > > > Is there another local cache to be configured in Infinispan to avoid users to re-login after restarting Keycloak? > > > It would be very helpful if you could give us a direction to investigate as rolling out several updates per day would lead to several re-logins from a user perspective. > > > Cheers, > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.jboss.org_mailman_listinfo_keycloak-2Ddev&d=DwICaQ&c=8G9fL-FODW3I-fESjFvsaw&r=BhQIS0sALperhBRJRNzxM3Ax9g4k8rNIuzNmM35TMgE&m=3avD19J-uUP_iVkM4EraHy9h5WmDyrGADDKcAsOq4Ns&s=Q_j6p3AqAYL7O0AhoDXIPelcAFrTksTbVZggtrcJ8wE&e= From william_jones20 at outlook.com Thu Aug 30 11:16:35 2018 From: william_jones20 at outlook.com (William Jones) Date: Thu, 30 Aug 2018 15:16:35 +0000 Subject: [keycloak-dev] Implementing user self service via REST APIs Message-ID: Hi As part of my Keycloak implementation, I would like to offer user self service, e.g. change password. I do not wish to theme the built-in user account dashboard, but instead build the functionality directly into our website. We are already using the Admin API for user administration, but as I understand it, it would not be appropriate to use this for actions which are actually being carried out by the end user. The auditing would be incorrect, and certain functionality is unavailable anyway - for example, whilst we could set a new user password via the Admin API, we would have no way of verifying that the existing user password is correct (we want the user to provide existing and new, as per the user account dashboard). As such, is the correct approach to this for us to extend KeyCloak with a set of custom REST endpoints to be called by an end user rather than an admin? They will be authenticated at this point so we will be able to pass down their access token for the authentication. If so, I assume I should be following the instructions under "Add custom REST endpoints" detailed at the following URL? https://www.keycloak.org/docs/3.0/server_development/topics/extensions.html Extending Server | Keycloak Documentation This is a very powerful extension, which allows you to deploy your own REST endpoints to the Keycloak server. It enables all kinds of extensions, for example the possibility to trigger functionality on the Keycloak server, which is not available through the default set of built-in Keycloak REST endpoints. www.keycloak.org Thanks William From pnalyvayko at agi.com Thu Aug 30 11:23:03 2018 From: pnalyvayko at agi.com (Nalyvayko, Peter) Date: Thu, 30 Aug 2018 15:23:03 +0000 Subject: [keycloak-dev] OAuth2 extension grants In-Reply-To: <707073d4-f582-629b-d5a5-017d91aac3c2@redhat.com> References: , <707073d4-f582-629b-d5a5-017d91aac3c2@redhat.com> Message-ID: Hi, Are there any plans to enable extending the existing OIDC protocol with OAuth2 extension grants (similar to http://docs.identityserver.io/en/release/topics/extension_grants.html)? Correct me if I am wrong, but it seems that the only way to add an OAuth2 extension grant type now is to add a new OIDC based protocol and create a derivative of protocol.oidc.endpoints.TokenEndpoint Cheers, Peter From dpalmer at redhat.com Thu Aug 30 13:58:03 2018 From: dpalmer at redhat.com (Douglas Palmer) Date: Thu, 30 Aug 2018 10:58:03 -0700 Subject: [keycloak-dev] Digital fingerprints Message-ID: <5FB93EB1-3BA0-4CF1-A381-94F28F06CC3E@redhat.com> Hi As part of KEYCLOAK-7416 I need to gather information about the device for each session. We are aiming to produce this page: P55-Device-Activity-3 . However, this requires more data about the client device than is carried in a user-agent header such as this: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.7 (KHTML, like Gecko) Version/9.1.2 Safari/601.7.7?. I?m thinking we will probably need to use a digital fingerprint library, probably executed on the client. Does anyone have a recommendation of a good library? Or have a better suggestion? Regards Doug From bruno at abstractj.org Thu Aug 30 15:37:49 2018 From: bruno at abstractj.org (Bruno Oliveira) Date: Thu, 30 Aug 2018 16:37:49 -0300 Subject: [keycloak-dev] Digital fingerprints In-Reply-To: <5FB93EB1-3BA0-4CF1-A381-94F28F06CC3E@redhat.com> References: <5FB93EB1-3BA0-4CF1-A381-94F28F06CC3E@redhat.com> Message-ID: The only thing which I can think right now is Webauthn (https://www.w3.org/TR/webauthn/). From what I see it's supported in the most popular browsers https://caniuse.com/#search=webauthn. Honestly I never tried, but maybe worth to take a look. On Thu, Aug 30, 2018 at 4:20 PM Douglas Palmer wrote: > > Hi > > As part of KEYCLOAK-7416 I need to gather information about the device for each session. We are aiming to produce this page: P55-Device-Activity-3 . However, this requires more data about the client device than is carried in a user-agent header such as this: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.7 (KHTML, like Gecko) Version/9.1.2 Safari/601.7.7?. > > I?m thinking we will probably need to use a digital fingerprint library, probably executed on the client. Does anyone have a recommendation of a good library? Or have a better suggestion? > > Regards > Doug > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -- - abstractj From Sebastian.Schuster at bosch-si.com Fri Aug 31 09:46:12 2018 From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST/ESY1)) Date: Fri, 31 Aug 2018 13:46:12 +0000 Subject: [keycloak-dev] Possible feature: role attributes In-Reply-To: References: <10b624a57cea40a1bcc9bd77b63b398a@bosch-si.com> Message-ID: <365a8b409e5248908b2eb312a6d202bc@bosch-si.com> All right. I would like to create a prototype for this. I would take inspiration from the way custom group attributes are currently implemented. I guess changes would be necessary in the following areas: ? DB schema ? Persistence layer ? Caching layer ? CRUD API ? Admin console ? Admin CLI ? Java client ? Admin events Anything I missed? Thanks and best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn From: Stian Thorgersen Sent: Montag, 27. August 2018 13:49 To: Schuster Sebastian (INST/ESY1) Cc: keycloak-dev Subject: Re: [keycloak-dev] Possible feature: role attributes I don't think we need to consider adding role attributes to the token. That would very quickly bloat tokens. I would like to see a bit more general use of role attributes as part of incorporating such a feature. Otherwise it would end up being a rather hidden feature. Some ideas I have in mind: * Ability to do crud of role attributes in admin console * Ability to query for roles based on attributes For future work it would be great to have attributes on everything. That would allow us to do something like OpenShift `oc` does. Where you're able to search and delete everything based on attributes. One nice use-case here is that you can tag all clients, roles, etc.. that belong to a deployment (a group of apps and services) and be able to view everything that is related to the deployment in Keycloak. On Mon, 27 Aug 2018 at 13:32, Schuster Sebastian (INST/ESY1) > wrote: Hi everybody, We have a use case where we would like to store additional meta-information for roles. This come from our IAM-requirements, that say there is a single responsible person for a role or that roles give access to data with different classifications. One way to store this kind of information would be to introduce role attributes to client and realm roles, basically similar to user or group attributes. For us, it would be sufficient to have this information purely as metadata, i.e. we would only read it through the audit log to inform the responsible person about role assignments if a role with a certain classification is assigned. In contrast to that, you can add group und user attributes to a token using user attribute mappers and the client application can extract this information from the token and act on it. WDYT? Does anybody else have similar requirements? Would you need role custom attributes also in the token? I can imagine that it gets kind of difficult to identify where attributes come from, once there are user, group, and role attributes, possibly with inheritance/composition. Best regards, Sebastian Mit freundlichen Gr??en / Best regards Dr.-Ing. Sebastian Schuster Engineering and Support (INST/ESY1) Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From hmlnarik at redhat.com Fri Aug 31 12:38:11 2018 From: hmlnarik at redhat.com (Hynek Mlnarik) Date: Fri, 31 Aug 2018 18:38:11 +0200 Subject: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final In-Reply-To: References: <3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com> Message-ID: Hi Thomas, the issue with repeating a query should be addressed by [1]. Out of curiosity, I wonder whether you'd have some spare time to test this change in your environment? Thanks --Hynek [1] https://github.com/keycloak/keycloak/pull/5525 On Thu, Aug 30, 2018 at 12:58 PM Hynek Mlnarik wrote: > Hi Sebastian, Thomas, > > I agree that this query is bad performance-wise. It is caused inability of > database to easily limit rows in the resultset of query that joins users > and attributes (and other tables) if there might are several attributes per > user while we want to return exactly n users (the LIMIT/OFFSET won't work > here for parent table due to joining with child records). Hibernate thus > resorts to its own filtering of the full resultset. That is one option, the > other is having subsequent selects and limiting batch size. Both approaches > have pros and cons, and actually the subsequent selects were changed to > subselects precisely because of performance. Looks like we might need to > revert back to the select approach and tweak batch size. That will require > some time for optimizations. > > For now, the most painful issue with user list seems resolved and thus I'm > merging the PR. There will be subsequent work though on further > optimizations to user / user list retrieval, that relates to both your > suggestion and Thomas's comment (that relates to authz resource server > store caching). Thank you both for your help! > > --Hynek > > > On Thu, Aug 30, 2018 at 12:06 PM Schuster Sebastian (INST/ESY1) < > Sebastian.Schuster at bosch-si.com> wrote: > >> Hi Thomas, >> >> >> >> Your query is the same as before, it is the initial retrieval of the user >> entity. The query I am referring to is the one loading the user attributes >> after the user entity has been fetched as part of >> ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel >> realm, UserModel user). It just looks wrong to me that all users are >> fetched when only the attributes of a single user are needed. However, with >> Hynek?s change, the user attributes are no longer fetched because only a >> BriefRepresentation is requested, that?s why it is faster now. I still >> think a full representation should also be faster and not fetch all users >> from a realm. >> >> >> >> Best regards, >> >> Sebastian >> >> >> >> Mit freundlichen Gr??en / Best regards >> >> >> *Dr.-Ing. Sebastian Schuster * >> Engineering and Support (INST/ESY1) >> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> GERMANY | www.bosch-si.com >> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> Sebastian.Schuster at bosch-si.com >> >> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. >> Stefan Ferber, Michael Hahn >> >> >> >> *From:* Thomas Darimont >> *Sent:* Mittwoch, 29. August 2018 22:55 >> *To:* Schuster Sebastian (INST/ESY1) >> *Cc:* Hynek Mlnarik ; Stian Thorgersen < >> stian at redhat.com>; keycloak-dev >> *Subject:* Re: [keycloak-dev] Potential performance regression between >> 3.4.3.Final and 4.3.0.Final >> >> >> >> Hello Hynek et al., >> >> >> >> I gave your PR 5517 a quick spin and tested the user listing in the >> admin-console on my Dell XPS (3.4GHz, 32g) laptop with local PostgreSQL >> backend. >> >> I can confirm that the response times for loading a single page in the >> user-listing are much better now with your fix: >> >> In a test realm with 10k users, loading a single page takes ~15ms >> compared to 8 seconds (!) before. >> >> >> >> The new query that is now generated whilst browsing the user list looks >> like this: >> >> >> >> Hibernate: >> >> select >> >> userentity0_.ID as ID1_75_, >> >> userentity0_.CREATED_TIMESTAMP as CREATED_2_75_, >> >> userentity0_.EMAIL as EMAIL3_75_, >> >> userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_75_, >> >> userentity0_.EMAIL_VERIFIED as EMAIL_VE5_75_, >> >> userentity0_.ENABLED as ENABLED6_75_, >> >> userentity0_.FEDERATION_LINK as FEDERATI7_75_, >> >> userentity0_.FIRST_NAME as FIRST_NA8_75_, >> >> userentity0_.LAST_NAME as LAST_NAM9_75_, >> >> userentity0_.NOT_BEFORE as NOT_BEF10_75_, >> >> userentity0_.REALM_ID as REALM_I11_75_, >> >> userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE12_75_, >> >> userentity0_.USERNAME as USERNAM13_75_ >> >> from >> >> USER_ENTITY userentity0_ >> >> where >> >> userentity0_.REALM_ID=? >> >> and ( >> >> userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null >> >> ) >> >> order by >> >> userentity0_.USERNAME limit ? offset ? >> >> >> >> The query looks reasonable with appropriate limits applied. >> >> >> >> However, it seems that policies are fetched for each returned user via a >> dedicated query, e.g.: >> >> >> >> select >> >> resourcese0_.ID as ID1_64_0_, >> >> resourcese0_.ALLOW_RS_REMOTE_MGMT as ALLOW_RS2_64_0_, >> >> resourcese0_.POLICY_ENFORCE_MODE as POLICY_E3_64_0_ >> >> from >> >> RESOURCE_SERVER resourcese0_ >> >> where >> >> resourcese0_.ID=? >> >> >> >> Would it be possible to load those in one batch? >> >> >> >> Cheers, >> >> Thomas >> >> >> >> Am Mi., 29. Aug. 2018 um 17:19 Uhr schrieb Schuster Sebastian (INST/ESY1) >> : >> >> Thanks Hynek, I?ll test this. I also tracked the issue down to the >> ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel >> realm, UserModel user) method. However, what I am also seeing is when the >> method is running, it is emitting SQL queries that are not scoped down to >> the user id of the user being transformed. E.g. for the user attributes of >> a user the following query is generated: >> >> >> >> select >> >> attributes0_.USER_ID as USER_ID4_72_1_, >> >> attributes0_.ID as ID1_72_1_, >> >> attributes0_.ID as ID1_72_0_, >> >> attributes0_.NAME as NAME2_72_0_, >> >> attributes0_.USER_ID as USER_ID4_72_0_, >> >> attributes0_.VALUE as VALUE3_72_0_ >> >> from >> >> USER_ATTRIBUTE attributes0_ >> >> where >> >> attributes0_.USER_ID in ( >> >> select >> >> userentity0_.ID >> >> from >> >> USER_ENTITY userentity0_ >> >> where >> >> userentity0_.REALM_ID=? >> >> and ( >> >> userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null >> >> ) >> >> ) >> >> >> >> The subselect returns all real users from the realm. It is neither paged >> nor bound to a specific user id. The same happens in the above method for >> credentials and required actions. I am not a JPA/Hibernate expert so I >> might be missing something here? >> >> >> >> Best regards, >> >> Sebastian >> >> >> >> Mit freundlichen Gr??en / Best regards >> >> >> *Dr.-Ing. Sebastian Schuster * >> Engineering and Support (INST/ESY1) >> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> GERMANY | www.bosch-si.com >> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> Sebastian.Schuster at bosch-si.com >> >> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B >> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. >> Stefan Ferber, Michael Hahn >> >> >> *From:* Hynek Mlnarik >> *Sent:* Mittwoch, 29. August 2018 16:35 >> *To:* Thomas Darimont >> *Cc:* Stian Thorgersen ; keycloak-dev < >> keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) < >> Sebastian.Schuster at bosch-si.com> >> *Subject:* Re: [keycloak-dev] Potential performance regression between >> 3.4.3.Final and 4.3.0.Final >> >> >> >> Thanks Thomas and all, >> >> >> >> great help with reproducing the issue at least for admin user list >> endpoint. Could you please test [1] that it indeed fixes the issue? It was >> caused obtaining too much user details when unnecessary (for user list, >> only few fields are necessary). >> >> >> >> As for increased memory requirements, these might be unrelated to this >> fix, and we'll keep an eye on this. >> >> >> >> --Hynek >> >> >> >> [1] https://github.com/keycloak/keycloak/pull/5517 >> >> >> >> >> >> >> >> On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont < >> thomas.darimont at googlemail.com> wrote: >> >> Hello, >> >> FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150 >> >> Cheers, >> Thomas >> >> Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen < >> sthorger at redhat.com>: >> >> > Can you open a bug for this in JIRA please and include as much details >> as >> > you can. >> > >> > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont < >> > thomas.darimont at googlemail.com> wrote: >> > >> >> Hi Sebastian, >> >> >> >> thanks for investigating! Unfortunately, I didn't have time yet to look >> >> further into the problem, but I also observed the slow attributes >> handling. >> >> Btw. we have around 100k users and ~700k attributes. The >> >> Entity-Attribute-Value data model takes it's toll here... >> >> >> >> @Stian so far I've only observed this when scrolling through the user >> >> overview in the admin-console, which also triggers >> >> the mentioned spike in heap usage. >> >> >> >> Cheers, >> >> Thomas >> >> >> >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian >> (INST/ESY1) >> >> : >> >> >> >>> Btw. we also have 40K user attributes... >> >>> >> >>> Best regards, >> >>> Sebastian >> >>> >> >>> Mit freundlichen Gr??en / Best regards >> >>> >> >>> Dr.-Ing. Sebastian Schuster >> >>> >> >>> Engineering and Support (INST/ESY1) >> >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | >> >>> GERMANY | www.bosch-si.com >> >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 | >> >>> Sebastian.Schuster at bosch-si.com >> >>> >> >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 >> B >> >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: >> >>> Dr. Stefan Ferber, Michael Hahn >> >>> >> >>> >> >>> >> >>> >> >>> -----Original Message----- >> >>> From: keycloak-dev-bounces at lists.jboss.org < >> >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen >> >>> Sent: Mittwoch, 22. August 2018 21:25 >> >>> To: Thomas Darimont >> >>> Cc: keycloak-dev >> >>> Subject: Re: [keycloak-dev] Potential performance regression between >> >>> 3.4.3.Final and 4.3.0.Final >> >>> >> >>> That's quite worrying. Is it limited to browsing users through the >> admin >> >>> console or are you seeing bad behaviour elsewhere? Same question >> applies to >> >>> the heap. My best bet here is that it has that something has changed >> around >> >>> user querying/caching. >> >>> >> >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont < >> >>> thomas.darimont at googlemail.com> wrote: >> >>> >> >>> > Hello Keycloak team, >> >>> > >> >>> > has anyone encountered some performance issues after upgrading 3.4.3 >> >>> > to 4.x (4.3.0)? >> >>> > >> >>> > Today I noticed a performance regression while preparing an upgrade >> >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment. >> >>> > >> >>> > In our test environment, we have around ~100k test users stored in a >> >>> > postgres-backed database. When we started the server with the new >> >>> > Keycloak version, the migration went through, and everything looked >> >>> > fine at first, but when we tried to browse the list of users via the >> >>> > admin-console, we noticed that the CPU and memory consumption of the >> >>> > server increased significantly, up to a point where Keycloak crashed >> >>> with an OOME. >> >>> > >> >>> > All previous Keycloak versions including 3.4.3 were very modest with >> >>> > their memory requirements and quite happy with ~1g heap. >> >>> > However, that seems to have changed in Keycloak 4.3.0 - there we >> >>> > needed at least 4g to prevent Keycloak from crashing with an OOME. >> >>> > Furthermore, we noticed that the response times for browsing the >> >>> > paginated user view increased significantly as well: >> >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In >> >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same >> operation >> >>> > takes >> >>> > ~7 seconds for a test realm with just 10k users. >> >>> > >> >>> > In the test realm with 100k users, the time to load a single page in >> >>> > the users listing was 66 seconds for version 4.3.0, on average - >> >>> > compared to quite stable 80ms in 3.4.3. >> >>> > >> >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5 >> >>> > seconds for 100k users, so I assume the processing logic in Keycloak >> >>> > is the culprit. >> >>> > >> >>> > The problem of long load-times can be reproduced with the Keycloak >> >>> > docker images and the in-memory database. I also created a small >> >>> > example project that creates some users with just a few attributes >> in >> >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres >> >>> > database to reproduce the problem. >> >>> > https://github.com/thomasdarimont/kc-user-regression-tester >> >>> > >> >>> > Cheers, >> >>> > Thomas >> >>> > _______________________________________________ >> >>> > keycloak-dev mailing list >> >>> > keycloak-dev at lists.jboss.org >> >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >>> > >> >>> _______________________________________________ >> >>> keycloak-dev mailing list >> >>> keycloak-dev at lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >>> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >>